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Abstract 


Public involvement in research occurs when the public, patients, or research participants are actively 
contributing to the research process. Involving people in research improves the quality, relevance 

and equity of research for everyone. Public involvement has been acknowledged as a key priority for 
prominent human genomics research initiatives in many different countries. However, to date, there 
has been no detailed analysis or review of the features, methods, and impacts of public involvement 
occurring in human genomics research projects worldwide, and evidence about the best methods to 


involve people remains limited. 


This thesis explores how we can strengthen the principles of equity and human rights in genomics 
research, using evidence-informed methods. This doctoral research initially explored how people 
have been involved in human genomics research. | published a global review of nearly 100 current 
international genomics research projects and discovered that only one-third of them reported 
involving people. Applying learning from this, | co-designed research with four different groups to 
explore and evaluate practical ways of involving people. This represented a program of participatory 
action research across four different projects and domains of human genomics, each of which had 
different implications and challenges with regard to involving people the research cycle. The 
participatory action research projects included a large cohort study of >15,000 healthy, elderly 
research participants, a group of patients and families affected by a rare immunological disorder, an 
extended family of donor-siblings who share the same sperm-donor father, and the Australian 


Indigenous Precision Medicine project. 


Involving each group had different challenges with regard to involving people the research cycle. To 
describe methods and assess impact, | created a standardised way to report involvement across the 
four studies. | led an international team of more than 40 people to develop ‘Standardised Data on 
Initiatives’ (STARDIT), which is now being used by multiple organisations including Australian 


Genomics and the Wikimedia Foundation open-access journals. 


Learning from this research indicates that people want to be involved and want to make decisions 
about genomics research and their own data, but they need support to get involved. In addition, 
researchers need support to involve multiple stakeholders in designing evidence-informed ways of 
involving people. Data from STARDIT can be used to support evidence-informed methods of 
involvement and strengthen equity in genomics research, helping make sure the benefits of 


genomics research are for everyone. 
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the thesis. This thesis has not been submitted for the award of any degree or diploma in any other 


tertiary institution. 
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Plain English Summary 


All life is made from code, from DNA. Human genomics research is the study of all the DNA in 
humans (the genome), together with the technologies that allow it to be analysed. When learning 
from genomics research is applied to human health, it is called genomic medicine, and it can 


improve lives. 


In the next five years, it is estimated that nearly two billion people worldwide will have had their 
DNA sequenced. How can we ensure that future genomic research benefits everyone? How can we 
avoid a future where only those who can afford it are able to access those benefits? How can we 


involve people in genomic research, to maximise public health benefit? 


Involving people in genomics research means sharing power, giving the public, patients and health 
technology consumers more control around decision making. Involving people is the best way to 
ensure genomics initiatives reflect the diversity of priorities in populations. It has been widely 
demonstrated that involving people in research improves the relevance and equity of research. 
Involving people increases public trust, acceptability and participation in genomics. It can also 
improve recruitment, ethical oversight and ultimately the quality of research outcomes. If genomics 
initiatives do not align with people’s values, there is a risk that entire populations (in particular those 
people who are at greater risk of exploitation or under-represented, such as Indigenous peoples) will 
not participate in future initiatives, affecting the potential positive impact of genomics for decades. 
While funding bodies are increasingly requesting evidence of how people have been involved, data 


for evidence-informed ways of involving people in genomics research is lacking. 


| am a public health researcher looking at genomic research and involving people in it. My PhD thesis 
explores how we can strengthen the principles of human rights in genomics, including using 
participatory action research. | have explored how can we do that practically, and what evidence is 
there about the best ways to involve people in other types of research, which can be applied to 
human genomics? For example, more people are discovering that a variation in their DNA might 
contribute to their risk of a disease, or that they or a loved one might pass on that risk to their 
children. Some people are told that the disease they are at risk of is so rare that it has not been well- 
researched, that there’s no treatment or that treatment is too expensive. Some people with very 
rare DNA variations feel that finding a community of other people with the same variation is their 
only hope of surviving the disease, by working together to improve research and treatments. 
However, the best way for people to get involved in shaping the future of genomics research is not 


always clear. 


As part of this doctoral research, | have published a global review of more than 100 current 
international genomics research projects and discovered that only one-third of them report 
involving people. Applying learning from this, | co-designed research with four different groups to 
explore and evaluate practical ways of involving people. This represented a program of research 
across four different projects and domains of human genomics, each of which had different 


implications and challenges with regard to involving people the research cycle. 


| worked with participants in one of the largest clinical trials in Australia, to co-design a multi- 
generational study that explored the preferences of multiple stakeholders, including how they’d like 
to be involved in the trial. | also worked with a group of people affected by a rare disease, helping 
explore how they would like to be involved in future genomics research using online discussions. 
After starting my PhD, | decided to have my own DNA tested and discovered that my biological 
grandfather was a prolific sperm donor who fathered up to 1,000 people. After consultation with 
ethical experts and a co-design process with the siblings, | worked with 20 biological relatives from 
this group to co-design online discussions about future genomic research. Finally, | also worked with 


remote Aboriginal communities to co-design genomics research protocols. 


Learning from this doctoral research has shown that people want to be involved and want to make 
decisions about their own data, but they need support to get involved. It’s also demonstrated that 
researchers need to involve people in designing involvement plans, using evidence-informed 


methods to do this. 


While I’ve learned that involving people has positive impacts, evidence about the best methods 
remains limited. For that reason, | created a standardised way to report involvement and I’m now 
leading an international team of more than 40 people to develop ‘Standardised Data on Initiatives’ 
(STARDIT). The working Beta version of STARDIT can report on research initiatives around the world 
in multiple languages. It has already been used by Australian Genomics to report planned work 
exploring how to involve people in genomics research. STARDIT can help us answer the question, 
‘What is the best way of involving everyone in shaping future research?’ Once we can answer this 


question, we can all be involved in making sure the benefits of genomics research are for everyone. 
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Executive summary 


Background 


Human genomics research is the study of all the DNA in humans (the genome), together with the 
technologies that allow it to be analysed. When learning from genomics research is applied to 
human health, it is called genomic medicine, and it can improve lives. Public involvement in research 
occurs when the public, patients, people affected by genomic variations of known or significance or 


research participants (hereafter ‘people’) actively contribute to the research process. 


There is evidence that involving people in research assists with recruitment, improves its quality, 
relevance and acceptability, and promotes equity of research. Involving people in genomics research 
means sharing power by giving the public, patients and participants control of aspects of the 
research — such as data access. In many countries, public involvement in human genomics research 
initiatives has been acknowledged as a key priority. However, to date, there has been no detailed 
analysis of the features, methods and impacts of public involvement in human genomics research, 


and no examination of practical, evidence-informed ways to involve people. 


Involving people in genomics research (and in population-wide genomic initiatives) is essential for 
public trust, support, funding, acceptability and participation. Involving people is also the best way 
to ensure that genomics initiatives reflect the diverse priorities of populations. In many parts of the 
world including Australia, demonstrating evidence of how people will be involved in genomics 
initiatives is now part of the requirements for funding applications. If genomics initiatives do not 
align with people’s values, there is a risk that entire populations will not participate in future 
genomic research. This would undermine the potential positive impacts of genomics for improving 
human heath for decades to come, especially in the case of under-represented populations, or 
populations at greater risk of exploitation, such as Indigenous peoples. Although people are involved 
in genomics to a variable extent around the world, there is currently no standardised way to plan, 
report or evaluate how they are involved. To ensure that power is shared in culturally appropriate, 
culturally safe, transparent and cost-effective ways, those planning and funding future initiatives 


require new evidence-informed methodologies. 


Research aims 


The research aims for this doctoral research are to: 


understand when and how people have been involved in human genomics research to date, 
and identify gaps that need to be addressed with new approaches and methods for 


involvement 


apply a participatory action research method to human genomic research, using four case 


studies, in order to learn more about the process of involving people in genomic research 


develop a standardised way of planning, reporting and evaluating involvement in order to 


improve future genomics research. 


Research methods 


This doctoral study was conducted in a public health research setting, as distinct from an ethical, 


social or psychological research setting. The work is intended to be pragmatic, while also developing 


and applying theoretical frameworks. Several reviews were undertaken, including a systematic 


scoping review of public involvement in genomics research. These reviews have informed the case 


study methodology used on the subsequent four case studies. The case studies used participatory 


action research methods to explore the views and perspectives of four different groups of people 


associated with different human genomics research studies. The thesis describes the participatory 


action research process used during the involvement activities, and the subsequent co-design and 


implementation of the studies, as well as their associated impacts. 


The case studies used different methods to involve people in every stage of the research cycle, 


supported by learning and development resources shared in different formats. Two used an online 


text-based discussion platform, two used face to face meetings, and one also used telephone 


interviews. The different methods used, and any associated impacts were reported in a standardised 


way. 


Case studies 


1. 


Involving research participants from Australia's largest clinical trial and cohort study of more 
than 15,000 healthy, elderly research participants, to co-design a future multi-generational 


research study (ASPirin in Reducing Events in the Elderly study — ASPREE) 


Involving people affected by a rare genetic disorder in shaping future research through working 


in partnership with a rare disease charity (ausEE study) 


Co-designing genomics research with one of the largest known groups of donor-conceived 


siblings in the world (Shared Ancestry study) 


4. Involving Australian Indigenous peoples in co-designing a research protocol for a precision 


medicine project (Indigenous Precision Medicine project) 
Standardised Data on Initiatives (STARDIT) 


This thesis also describes the co-creation of Standardised Data on Initiatives (STARDIT), a 
standardised way of reporting on the planning, execution and evaluation of involving people in 
human genomics research other initiatives. | conceived STARDIT and led an international co-design 
process to co-create this new framework. During the co-creation process | applied it to the four case 
studies. | used the STARDIT framework for reporting on the participatory action research process 
used in the four case studies from different prominent genomics research projects in Australia. The 
framework was also used to create standardised data for a cross-case analysis of preferences for 


involvement, methods of involvement, and the impacts of involvement across the four case studies. 


Results 


This thesis has developed standardised ways of planning, reporting on and evaluating stakeholder 
involvement in genomics research and other related initiatives. Learning from this research will help 


advance the field of involving people in genomics research. 


Results from reviews 


The narrative review demonstrated there is not currently enough data to complete a meta-analysis 
of quantitative or qualitative data about involvement in genomics research. It informed the decision 
that a systematic scoping review was the most appropriate method to search for relevant data. 
Findings from this review also suggested that methods of involving people guided by the paradigm 


of participatory action research were most likely to have impacts. 


The systematic scoping review provided a useful ‘snapshot’ of current international genomics 
research projects, by using a database provided by the Global Alliance for Genomics and Health 
(GA4GH). While a third of initiatives reported involving people, only 10% of initiatives reported 
impacts. The limited reporting of involvement suggested there would be intrinsic value in developing 
a more systematic method of both reporting and evaluating how people are involved in human 


genomics research, as data from such reporting could provide the evidence required to inform 
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future policy around involvement of the public in genomics research. The recommendations from 


this review informed the co-creation of STARDIT. 
Results from Standardised Data on Initiatives (STARDIT) 


STARDIT provides a standardised ways of planning, reporting and evaluating genomics research, 
including involvement. During this doctoral research, | co-created Standardised Data on Initiatives 
(STARDIT), a standardised way of reporting on involvement in genomic research and initiatives. The 
STARDIT system enables reporting in multiple languages and is applicable beyond public health 
genomics. It has already been used by projects beyond those described here, including by the 
Australian Genomics working group ‘Involving Australia’, and has been recommended for use in 


describing involvement in biobanks. 
Results from case studies 


As part of this doctoral research, STARDIT was also successfully used to map preferences (using the 
preference mapping tool STARDIT-PM), plan involvement and report the impacts of involving 
stakeholders across the four case studies (both research participants and study team members). In 
each case study, the process of involving people in the research led to positive impacts and 
outcomes. Using STARDIT allowed these to be reported in a standardised way, while using STARDIT 
for the analysis made it possible to combine all the public domain data from the case studies and to 
categorise the data. This allowed comparison of STARDIT-PM data from 83 stakeholders in the three 
case studies in which preferences were mapped (by the facilitators of the online discussions in the 
ausEE and Shared Ancestry case studies, and by study team members from the ASPREE case study). 
By combining data about preferences for who should be involved in research from the Shared 
Ancestry and ausEE case studies, it was possible to show that 45 per cent of participants’ responses 
‘widened’ to include a preference for more people to be involved in the research (N=43/95). For 43 


per cent, preferences stayed the same (N=41/95), and in 12 per cent they ‘narrowed’ (N=11/95). 


Overall, my results suggest there is inherent value in planning and reporting involvement ina 
standardised way, which allows the creation of data from which to make evidence informed 
decisions about effective ways of involving people. In this thesis, | have led the co-creation process 
to build a way of doing this that anyone can use, and demonstrated how it can be applied in multiple 
research settings as case studies. By reporting different case studies in a standardised way, | have 
shown how it is possible to combine the data in order to better understand data from multiple 
sources. In this thesis, | used this data to show that through the process of involving people in the 
genomic research, most participants’ preferences for who should be involved in genomics research 


‘widened’ to include more people. 
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Discussion 


Learning from these case studies can be applied to various research settings, but is particularly 
valuable for research involving populations at greater risk of exploitation— such as people affected 
by rare diseases or Indigenous populations, where stakeholder involvement is critical. The published 
outputs from this thesis have collectively received more than 43 citations, demonstrating the 
interest in the issues explored in this thesis. During this thesis STARDIT has been used by projects 
outside the scope of this doctoral research, including by a working group led by Australian 


Genomics, to report prospectively on how people will be involved. 


In order to ensure that planning, reporting and evaluation of involvement in research increases 
equity in public health genomics and other fields, further ongoing co-development of the methods 


described here (including STARDIT) is required. 
Conclusion 


Greater involvement of stakeholders in global human genomics research has intrinsic value 
worldwide. In this thesis, | reviewed contemporary practice and applied theoretical constructs in the 
context of real-world genomic research to develop novel ways of reporting impacts, described in 
peer-reviewed case studies. My thesis has demonstrated that STARDIT can be used as a standardised 
way of planning, reporting and evaluating involvement in genomics research, which works across 
multiple human languages. STARDIT has been demonstrated as a systematic, practical and effective 
way to co-design, report on and evaluate public involvement in genomics research. The data created 
by STARDIT has potential to inform best practice in future genomics research, and other disciplines. 
Further work to co-develop more systematic ways of reporting and evaluating such involvement, 


and supporting ways of embedding this practice into research would be highly beneficial. 
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Chapter 1 — Introduction 


In order to improve evidence-informed methodology, this doctoral thesis explores why people 
should be involved in genomics research, how they can be involved, and demonstrates ways of 


planning, reporting and evaluating involvement in a standardised way. 


The research aims for this doctoral research are: to (1) understand when and how people have been 
involved in human genomics research to date; (2) apply a participatory action research method to 
human genomic research, using four case studies, in order to learn more about the process of 
involving people in genomic research; (3) develop a standardised way of planning, reporting and 
evaluating involvement in order to improve future genomics research. The research aims and activity 


are summarised in more detail in Chapter 2 ‘Research aims and activity’. 


Introduction to involvement in health research 


What is public involvement? 


In 1969 Arnstein wrote that, “Participation of the governed in their government is, in theory, the 
corner-stone of democracy”.’” Less than a decade later, this sentiment was enshrined in the World 
Health Organization’s (WHO) ‘Declaration of Alma-Ata’, which stated that “the people have the right 
and duty to participate individually and collectively in the planning and implementation of their 


health care.”*® The Alma-Ata declaration was followed by this statement from the WHO in 1986: 


“Health promotion works through concrete and effective community action in setting 
priorities, making decisions, planning strategies and implementing them to achieve better 
health. At the heart of this process is the empowerment of communities — their ownership 
and control of their own endeavours and destinies. Community development draws on 
existing human and material resources in the community to enhance self-help and social 
support, and to develop flexible systems for strengthening public participation in and 
direction of health matters. This requires full and continuous access to information, learning 


opportunities for health, as well as funding support.” ? 


Involvement should attempt to create a balance between individual autonomy and collective values 
in order to achieve equitable access to the things which improve health and wellbeing.” Variation in 
people’s priorities means that it is important to try to articulate the differences and decision-making 


processes in a transparent way.”° ~? The process of involvement must be as inclusive as possible. For 
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example, people must feel that they have been supported to influence the creation of health policy 


or services to ensure that they are acceptable.2°3 


While there is no international consensus on terminology to describe involvement in health 
research,” the United Kingdom’s National Institute of Health Research (NIHR) provides one of the 


most succinct and helpful definitions of involvement: 


“By public involvement we mean research being carried out ‘with’ or ‘by’ members of the 


public rather than ‘to’, ‘about’ or ‘for’ them,””° 


As part of an international review in 2017, eight government research funders produced guidance 
that endorses patient and public involvement.”* However, while international support for 
involvement is growing, definitions of ‘involvement’ are varied and inconsistent. Even between and 
within Anglophone countries, words such as ‘engagement’, ‘involvement’ and ‘participation’ are 


used interchangeably.2 28 


Similarly, to be meaningful, words such as ‘patient’, ‘public’, ‘community’, ‘consumer’ and ‘payer’ 
need careful articulation. While terms such as ‘people’ or ‘the public’ exclude fewer people than the 
terms above, they also lose some meaning in the process of generalising. What is meant by ‘the 


public’ is defined more clearly by the UK Health Research Authority which states: 


“... the term public means patients, potential patients or members of the public including 
those with known genetic dispositions, carers and people who use health and social care 
services as well as people from organisations that represent people who use health and 


social care services.” 2°° 


Noting that ‘potential patients’ includes everyone, the word ‘stakeholders’ can be a more useful 
term, if the usage of this term is defined carefully. In this thesis, the term ‘stakeholder’ means 
anyone who has a ‘stake’ in health research, in particular those with important knowledge, 
experiences, expertise or views that should be taken into account.?®3%31 It can include, as sub- 
categories; researchers; research funders; policy makers, people affected by the research; people 
with specific health conditions, people with specific genomics variations; patients and the general 
public (who may also be categorised as ‘tax-payers’ for publicly funded research). The term ‘people’ 


and ‘the public’ will be used to refer specifically to patients, potential patients, carers, payers, 
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consumers of health technology and the general public, excluding professional researchers, research 


funders, policy makers and anyone else with a professional connection to research. 


However, current variations in terminology and inconsistency in definitions in all human languages 
make it challenging to compare how different stakeholders are involved in different tasks in 
genomics, and why.2”2 This creates significant problems in public health genomics, impedes 
comparison and the ability to share standardised data to support both ethical and evidence- 


informed methodologies. Ways of exploring the ‘why’ of involvement are explored in detail in the 


section ‘Research Paradigms’. 


Why involve people in research? 


Involving people in shaping future genomics research can be important for a range of reasons. There 
is evidence that involving multiple stakeholders (including the public and patients) in research 
improves research and outcomes. Research prioritisation processes that do not involve people can 
result in a mismatch between the research and the needs of people the research is intended to 
help. Involving people in the research cycle helps improves trust in research and public influence 
over research.2°3*35 It can help ensure that research is conducted in an ethical, accessible, 
responsible and transparent manner.?® It can also help ensure that research reflects the balance and 
diversity of priorities within populations.*®?’ Involving people in research can therefore be 


considered both a moral and a scientific imperative.2?“° 


Increasing the involvement of the public in human genomics research (hereafter ‘genomics 
research’) and genomics policy development has been identified as a crucial aspect of responsible 
research practice.”°*" For genomics research to be successful, public support, in terms of funding 
decisions and willingness to participate, is necessary.** This is particularly important with respect to 
populations at greater risk of exploitation, such as those with histories of experiencing medical and 


research abuse.” 


The principles and best practices of research involvement outlined above have not yet been fully 
integrated into the emerging field of human genomic research. Using an evidence informed 
methodology, my thesis seeks to help improve the translation of these principles into the field of 


human genomics, where they are highly relevant and urgently required.*?+ 
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What is human genomics research? 


Human genomics research is the study of all the DNA in humans (the genome), together with the 
technologies that allow it to be analysed. This analysis includes interactions of the DNA with other 
molecules and with the environment. When learning from genomics research is applied to human 


health, it is called genomic medicine, and it can improve and save lives. 


Genomics research has the potential to improve disease prevention and to inform more targeted, 
more effective interventions. Genomics research works best when it is combined with rich data 
about a person’s medical history, lifestyle and other information. With more data, we can make 
more accurate predictions. Around the world, more people are participating in human genomics 
research and it is impacting on more people’s lives in multiple ways. The number of people involved 
in genomics research is predicted to grow substantially in coming years.“ By 2025, it is estimated 
that nearly two billion people worldwide will have had their DNA sequenced. Responsible and 


effective public involvement in human genomics research is thus a global imperative.*° 


According to the Global Alliance for Genomics and Health, (as of March 2021)” there are currently 
221 active genomics initiatives globally, noting this is not an exhaustive list. These initiatives include 
biobanks, databases and other data sharing initiatives. Nine of these are government-led health 
initiatives. As the cost of sequencing, analysis and data storage declines, the number of global 
initiatives is predicted to grow exponentially, with large segments or even entire populations 


participating in such research. 


In addition, increasing numbers of people are purchasing online DNA tests from commercial 
companies and gaining information about their own DNA. Sometimes called ‘direct-to-consumer’ 
(DTC) or personal genomic tests (PGT), these can be broadly grouped into ‘health-related’ and 
‘recreational’ tests,’ with the latter including tests for wellbeing and ancestry purposes. Popular 
providers such as 23andMe and Ancestry DNA charge a one-off fee for the DNA test and result but 
claim the rights over the produced DNA data. DNA data is then used to improve their services and 
are often sold at a profit to other companies, which results in significant amounts of data for 
research.*? Private ‘direct-to-consumer’ companies hold some of the largest DNA databases in the 
world, with 23andMe and UK drug maker GlaxoSmithKline recently entering a partnership to identify 


potential candidates for clinical drug trials.°° The issue of informed consent in this process is 


explored in the Discussion section ‘Implications for ethics and participatory action research’. 
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Crucially, in addition to receiving data on their own DNA, many of these services offer people the 
choice of access to their own data, sharing their data with other projects or ‘donating’ data for 
research. Communities of people researching themselves are also redefining the relationship 
between ‘researcher’ and the researched. Open access and ‘community-led’ projects — such as 
GedMatch,°>? OpenSNP** and Open Humans*? — have potential to make access to interpretation of 
data and control over personal data more inclusive. However, these projects remain at risk of 
unauthorised uses of data and buy-outs — for example, despite being a not-for-profit volunteer 
initiated project, GedMatch has been used for high-profile criminal justice applications, and was 


recently sold to a venture capital firm.5 


It is important to note that while this thesis is concerned with human genomics research in the 
context of ‘public health’, human genomics research is not limited to this domain. Genomics 
research and the applications of genomic data are not limited to public health and medical 
applications, and the ‘blurring’ of the boundary between medical and non-medical uses has been 


noted in a recent report by the UN.°° 


For many members of the public, such distinctions are academic, and they primarily associate the 
practical application of genomics research to domains such as ancestry and criminal justice.°° 
Participants in two of the four cases studies in this thesis raised valid concerns about having data 
they provided for health research used by the criminal justice system. This issue is explored in the 


results section ‘Different interests’. 


In this thesis, the term ‘genomics research’ is used to encompass domains beyond health, 
government and academia. The term ‘genomic medicine’ or ‘public health genomics’ is used to 


provide specificity, when required. 
Research or care? 


Even within the domain of genomic research for ‘medicine’ and ‘health’, the conceptual and 
practical division between ‘healthcare’ and ‘health research’ is blurring, in particular, in the 
treatment of rarer cancers.” For some people with rarer cancers, participating in research is their 
only care option, so that from the patient perspective health research is also a ‘service’. There is also 
often a temporal gap between the discovery of effective new treatments in research settings and 
their availability in clinical practice.” This interval can create pressure on researchers to provide 
clinical care that is not available to patients elsewhere.” Thus, the distinction between genomics 
research and genomics medicine is increasing unhelpful. This is best illustrated by the 


comprehensive cancer centre model. 
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The USA’s comprehensive cancer centre model (also used in Australia) aims to facilitate the 
translation of research (including genomics) into routine clinical care.” Using learning from 
genomics research to carry out genomic profiling (of people’s DNA and tumour DNA) in order to 
inform medical treatment is known as ‘precision medicine’— the aim of which is “giving the right 
treatment to the right patient at the right time”.*° Similarly, ‘precision oncology’ resists definition as 
‘one thing’, and is more accurately an interconnected concept which is “erasing boundaries between 
the clinic, laboratory and healthcare industry”.©° Contemporary cancer centres blur distinction 
between research and health care as they must integrate data from clinical observations into 
laboratory research, develop clinical and public health intervention strategies from basic scientific 
discoveries, and provide high-quality treatment and diagnosis. In addition, they must develop 


educational resources about cancer prevention for the public. 


Precise terminology in ‘precision medicine’ thus remains a challenge, with some patients reporting 
that such terms are not ‘meaningful’.® Accordingly, in this thesis, involvement in human genomics 
research can include involvement in healthcare, health services and health research (including 
industry) where learning from genomics research is directly applied, or data from those activities 


inform genomics research. 
Genomics research and involvement 


“No sensible decision can be made any longer without taking into account not only the 


world as it is, but the world as it will be ...” (Isaac Asimov, 1978). 


The data generated by genomics research is literally astronomical in scale.® Projects such as the 
100,000 Genomes Project create hundreds of millions of data points, with global sequencing 


capacity expected to exceed 35 petabases per year. 


To be useful, such human genomic data require combination and comparison with similar sized 
datasets, including medical records and other phenotype data. As no single organisation worldwide 
has enough resources to fund research at the scale required for clinical genomics research, national 
genomics programs must collaborate of necessity.“ These collaborations require ethical standards 
and protocols for data sharing and provide an important opportunity for the public to be involved in 
shaping decisions about research. Memoranda of Understanding signed by Genomics England with 
British Columbia in Canada and the Garvan Institute in Australia are examples of such 


collaboration.© The Chief Scientist for Genomics England stated that this kind of data sharing and 
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collaboration provides an opportunity to “showcase their achievements to patients and the public in 


order to promote further involvement from those communities”. 


There is a growing need to increase public involvement in human genomics research and policy 
development and this has been identified as a crucial aspect of responsible research practice.?°*? As 
successful genomics research relies on people choosing to share their data, public support is 
essential. Many high-profile genomics research initiatives have already made public statements 
about the importance of involving people, with some governments positioning public involvement as 
a democratic right.?t3>67 For example, in the 2017 report ‘Generation Genome’, the UK's Chief 
Medical Officer suggested that shaping the future of genomics research requires the “active 
involvement of many stakeholders including patients, health professionals, researchers, 


policymakers, and wider society,” with a “key role for public engagement and involvement” 23 


Examples of public involvement in more recent projects are summarised in detail in the global 
systematic scoping review of public involvement in genomics (Chapter 4).? Initiatives which feature 
multiple ways for people to be involved include the UK Biobank, Genomics England and the USA’s 
Precision Medicine initiative ‘All of Us’. In Australia, the ‘Genomics Health Futures Mission’ (part of 
the Medical Research Future Fund) requires that the public should be “involved in setting the 
priorities” of the Mission.® In 2019, it became compulsory that grant applications demonstrate how 
people would be involved in proposed projects. Australia’s Medical Services Advisory Committee 
also seeks feedback on the health technology assessment (HTA) process from the public and people 


affected by genomic variations. 


Compared with other kinds of research, genomics research poses unique challenges in relation to 
involvement. Technological advances in genomics and the clear need for data sharing have seen a 
shift from autonomy towards concepts of reciprocity, solidarity and universality. 70271546) The need 
for data from so many people, who share approximately 99.9 per cent of their genome with all other 
humans, challenges concepts of individualism and promotes concepts of ‘solidarity’.’? The inherent 
commonality in the DNA we all share gives rise to an expectation that any benefits from knowledge 
generated by genomics research will be shared by all humans. In a 2015 report, the United Nations 


International Bioethics Committee wrote of concepts of justice and solidarity, stating that: 
“... genetics promises to offer an unprecedented contribution to improve health care. These 


advancements should be shared with society as a whole and with the international 


community; any discrimination has to be avoided.”*> 
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Concepts of individualism and solidarity are explored further in the Methods (Chapter 3) in the 


section ‘Individual rights or collective solidarity?’’. 


Genomics and other biomedical research present specific social, ethical and legal challenges, 
including those relating to communication and commercialisation of research results, balancing 
individual rights against the collective good, potential for genetic discrimination, and data 
protection. Emerging biotechnology should be subject to public debate about ethics and decision 
making, and orientated to the pursuit of the public good.**°*2) International cooperation is required 
to achieve real and effective public involvement in genomics research and innovation.*“"2) By 
involving the public in shaping future genomic research, we can create research that is more likely to 


benefit more people.’°°) 


However, those participating in either government or privately funded research often have limited 
information on which to base informed decisions about participation. This extends to information 
about what will happen with their data, control over their data once they are entered into 
databases, and opportunities to be involved in data-governance decisions.2” While innovations in 
giving research participants control over their data are being made,” the public still has limited 
control over research questions, priorities or methods, and limited ways to hold researchers who 
breach law or agreed ethical standards accountable.”’ In addition, there is limited evidence on ways 
to carry out collective decision making at various levels of genomics research governance — including 
in international law, government policy and other research initiatives. Evidence is also limited in 
relation to the effectiveness of education to improve genomics literacy (with both the public and 
health professionals) and to support people to critically appraise information sources about 


genomics research.” 


The social movements in biotechnology such as ‘Do-it-yourself Biology’ (DIY-Bio), warrant 
consideration in this thesis. Projects such as the Open Insulin project have highlighted the fact that 
access to medicine (including genomic medicine) is not universal or affordable for many people.” A 
University of Sydney project called ‘Breaking Good’, which worked with school children to create 
malaria drugs, indicates the growing acceptance of community-controlled research processes.” My 
own work with the charity ‘Science for All’ and the DIY-Bio charity ‘BioQuisitve’ on the ‘Wild DNA’ 
project showed how cheaper access to genomics sequencing and analysis is providing new frontiers 
for more people to get involved in doing research themselves. As more people are beginning to take 
direct action to solve issues of access to genomic medicine, the DIY-Bio movement is thus further 


blurring boundaries between ‘researchers’ and ‘the public’. This phenomenon is explored further in 


the Discussion section ‘DIY-Bio and health technology’. 
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The need for research into public involvement in 


genomics research 


Despite increasing interest worldwide in large-scale human genomics initiatives, limited research has 
been done on how the public are currently being involved, and no research has been conducted on 


the impacts and benefits of genomics initiatives involving the public. 


While systematic reviews on involving the public in other types of health and medical research have 
been published,?®””78 there are no comparable reviews in human genomics. In other areas of 
medical research, many reviews have concluded that public involvement activities are inconsistently 
reported or under-reported?83879-81 — including the specific ways in which people are involved in 


medical research and any impacts from involvement.24:7879 


Genomics research is a global enterprise and will need to continue to involve populations who speak 
diverse human languages. Accordingly, best-practice methodologies for involvement are needed and 
must work across these same boundaries. How such urgent challenges can be met is the subject of 


this thesis. 


These issues are explored further in the Methods section ‘Involvement in genomics: Mapping what is 
known and unknown’, and in Chapter 4 in the global systematic scoping review of involvement in 


genomics research that | conducted as part of this thesis. 


Research conducted as part of this thesis 


This thesis describes how | formed partnerships with two prominent genomics projects in Australia 
and two communities of people affected by genomic research in order to conduct participatory 
action research in these four settings. This thesis reports the processes as separate case studies, 
using a method of standardised reporting in order to report and compare the case studies 
consistently. The results and learning from these case studies is summarised in Chapter 9 ‘Results: 
Comparison of all case studies’ and Chapter 10 ‘Discussion’. This thesis concludes by summarising 
how the observations, learnings and outputs from this thesis have advanced the field of evidence 


informed involvement in genomics research. 
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About this thesis 


While this thesis focusses on involving people in the context of genomics research, it is a thesis 
about participatory action research, involvement in research and ways of planning, reporting and 
evaluating it in a standard way. Public health genomics was chosen as the area to apply these 
paradigms as it was identified as a complex area requiring urgent attention. Doctoral research for 
this thesis has been included in 8 reviewed (or under review) publications and 2 pre-prints. The 
published papers are included in this thesis in full or in part. To aid the examiners, peer-reviewed 
content is indicated by italic text before and after the peer-reviewed content. The section 
‘Publications and author contributions' summarises all publications associated with this thesis that | 
have authored or co-authored. This section outlines my contribution to each publication and shows 


how each is incorporated into, or has informed, this doctoral research. 


Text-based additional files from the published peer-reviewed articles can be found in the 
appendices, with the data files hosted by the journals, accessible via the Digital Object Identifier 


(DOI) links. 


In order to explain the iterative participatory action research processes used, the section ‘Thesis 
Timeline’ presents a visual timeline of the doctoral research and related activities. Visual summaries 
(infographics) of the research are also included, where possible, to make the content matter more 
accessible to those with different learning or processing preferences. The structure of this thesis is 
described in more detail below in the section ‘Thesis Structure’. To facilitate non-linear reading of 
this thesis on electronic devices, hyperlinks (denoted by an underscore) are included in chapters and 
subsections, along with a detailed table of contents which can be found in the section ‘Detailed 


contents’. 


Formatting and referencing styles are consistent throughout, including the peer-reviewed articles, 
meaning all referencing is consistent throughout this document. The text and graphical content of 
the peer-reviewed articles in this thesis are otherwise identical to the published versions, although 


numbering of figures and tables within peer-reviewed articles is aligned with the published versions. 
Formatted versions of articles created by the publishers can be found by following the hyperlinks in 


the references. This thesis uses the American Medical Association referencing style which uses 


superscript numbers.° 
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As this thesis has been written by one person (Jack Nunn), as the author, | will use a first-person 
active voice to describe things | have done. However, when they have been done as part of a team 
(for example, all the case studies), | will use the third person passive voice to describe what was 


done by the study team. 
Audience 


The audience for this thesis is anyone who is interested in learning more about evidence-informed 
ways of involving people in health research. Where possible, it has been written in plain English, with 
any specialist terms defined (see section ‘Definitions’). The thesis is thus written for a general 
audience with no specialist knowledge required, noting that the methods used in this thesis 
(including paradigms, data collection and analysis) are explained within the thesis, with references 


provided where further reading may be helpful for the reader. 


While this PhD focusses on involving people in genomics research, it is a thesis with relevant and 
transferable applications to participatory action research, and involvement in other kinds of research 
and ways of planning, reporting and evaluating such participatory research in a standard way. 
Genomics was chosen as the area of application for these methods as it was identified as a complex 
area requiring urgent attention. While a basic understanding of genomics is helpful, expertise in 


genomics is not required to understand this thesis. 
Definitions 


Words and related concepts used throughout this thesis are defined in Table 1.1. WikiData 
categories are used in order to standardise terminology, where possible. In some cases, | have 


created, amended or improved the Wikidata item.? 


Table 1.1: Defining words and concepts 


Word 


(Wikidata entry) 


Definition 


Barriers Barriers refers to things that might prevent something from happening. 
(Q109580927 


Enablers Enablers refers to things that help enable something to happen. For example, paying people’s 


(facilitators) | travel expenses might enable them to get involved in research. 
and barriers 


(Q109580938 


1 The Wikidata categories have been formatted as hyperlinks to the Wikidata item, as they are not in themselves references but links to 
structured data, which in some cases, | have edited the Wikidata online entry. For transparency, | have used my Wikimedia username 
‘JackNunn’ for all such edits. 
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Note: The word ‘facilitators’ is used in the published scoping review of public involvement in 


genomics, whereas ‘enablers’ is used throughout the rest of this thesis. 


Exploitation | The word ‘exploitation’ in this thesis means the act or result of forcibly depriving someone of 

(0859884) something to which they have a right, such as autonomy over the use of their own data. Note: 
The word ‘vulnerable’ was initially used in some published manuscripts of this thesis, however 
as a result of the co-design process of STARDIT and thesis review, it was decided that this word 
was problematic, as it is imprecise and vague about who is deciding who is vulnerable and why. 
82 Accordingly, the word ‘vulnerable’ was subsequently changed to ‘people/populations at 
greater risk of exploitation’ in this thesis and later publications. 

Genome A genome is an organism's complete set of DNA, including all of its genes. 

(Q7020) 

Genomics Genomics is an interdisciplinary field of biology which explores the structure, function, 

(02220148) evolution, mapping and editing of genomes. 

Genomics The term ‘genomics research’ is used in this thesis to describe research into genomics domains 

research beyond health and beyond government and academia. The term ‘genomic medicine’ or ‘public 
health genomics’ is used to provide specificity, when required. 

Involvement | Involvement in research refers to research being carried out ‘with’ or ‘by’ members of the 

in research public, rather than ‘to’, ‘about’ or ‘for’ them. This can include working to prioritise research 

(Q109581008 


topics, helping design research, helping manage it, and helping evaluate it.®? 


Participatory 


Participatory action research is an umbrella term which describes a number of related 


action approaches, including forms of action research which embrace a participatory philosophy and 
research include ‘co-design’ and ‘co-production’ of research.* It is a process whereby researchers, the 
e public and other relevant stakeholders “work together, sharing power and responsibility from 
the start to the end of the project”, including knowledge generation and translation.®° 
Peopleand | The term ‘people’ and ‘the public’ will be used to refer specifically to patients, potential 
‘the public’ patients, carers, payers, consumers of health technology and the general public, excluding 
Public professional researchers, research funders, policy makers and anyone else with a professional 
(Q2388316) 


connection to research. 


Stakeholder 
(Q109581040} 


The term ‘stakeholder’ means anyone who has a ‘stake’ in health research or service, in 
particular those with important knowledge, experiences, expertise or views that should be 
taken into account. It can include: researchers; research funders; policy makers; people affected 
by the research; people with specific health conditions; people with specific genomics 


variations; patients; and the general public (including ‘tax-payers’ for publicly funded research). 
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Thesis structure 


This thesis documents doctoral research which included multiple reviews and four case studies and 


was also informed by professional experience in parallel to the research. 


A timeline showing thesis development is provided to aid reader understanding of its distinct parts 


and how they interrelate (Figure 1.1). 


Chapter 1 introduces the concept of involvement and how it applies to genomics research. The 
current context of genomics research is summarised in order to show the importance and urgency of 


addressing the research questions examined in this thesis. 


Chapter 2 summarises the research aims of the thesis. 


Chapter 3 provides a detailed overview of the methods at each stage of the research process. It 
explores the perspectives, research paradigms and methods used and provides a summary of the 
research methods. The section ‘Involvement in genomics: Mapping what is known and unknown’ 
shows how the narrative review informed the choice of research paradigms, and how the scoping 
review informed the development of the ‘Standardised Data on Initiatives’ (STARDIT). This section 
concludes with a description of the co-creation process | led to produce STARDIT, the framework 
used throughout this thesis to describe the data from the case studies in a standardised way. The 
Beta version of STARDIT is included to explain the system which is central to the methods used in 


this doctoral research. The methods described in this section are further appraised in the Discussion 


Chapter, in section ‘Methodological evaluation’ 


Chapter 4 (results chapter) contains the published, peer-reviewed systematic scoping review of 


involvement in global genomics research projects. 


Chapters 5, 6 and 7 (results chapters) contain the three published, peer-reviewed case studies | led 


as part of this doctoral research. 


Chapter 8 (results chapter) is a reflective case study | authored, about my work with organisations 
involving Indigenous peoples in genomics research. The chapter describes the co-creation of a 
research protocol, and explores the co-design process from the very inception of a research project, 


as well as how to report intended involvement in a standardised way using STARDIT. 
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Chapter 9 summarises the results from all case studies and compares the findings from each. This 
cross-case analysis includes an exploration of the themes, generalisations, similarities and 
differences between these case studies. It uses STARDIT to compare the data in a standardised way 


using a combination of qualitative and quantitative data. 


Chapter 10 discusses the data presented in the previous chapters and summarises the main findings, 
outputs and dissemination of findings. It synthesises the main ideas explored in the thesis and 
summarises the new knowledge generated by this doctoral research. It summarises the implications 
of the doctoral research in the context of human genomics research and wider contexts. The chapter 


concludes with a series of recommendations informed by the learning from this doctoral research. 


Chapter 11 summarises this doctoral research and its conclusions. 


All research procedures reported in the thesis were approved by the relevant Ethics Committee, 


with relevant ethics information included in each chapter. 
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Figure 1.1: Thesis timeline 


2016 2017 2018 2019 2020 2021 
REVIEWS Preliminary research Narrative review Scoping review published Reviews for STARDIT 
CASE ASPREE Co-design Interviews and event Evaluation Publication 
STUDIES 

AusEE Co-Design Recruitment Discussions Evaluation Publication 


Precision Medicine for Aboriginal Australians ~ Co-Design — “Visit community "Published protocol - 
i 


jl STARDIT Standardised Data on Initiatives (STARDIT) conceived Alpha version Working Beta Submission 


Australian Genomics working group ‘Involve Australia’ votes to use STARDIT, October 2021 


Work and events which informed this thesis; out of scope of this project 


Work with health & citizen science organisations Strategy Liaison (Wiki Journals) MSAC Cochrane Council 


Started using Loomio Campfires and Science Wild DNA project 


= Received DNA results Founder and Director of charity ‘Science for All’ 


Chapter 2 — Research aims and activity 


The research aims for this doctoral research are to: 


1. understand when and how people have been involved in human genomics research to date, and 
identify gaps that need to be addressed with new approaches and methods for involvement 

2. apply a participatory action research method to human genomic research, using four case 
studies, in order to learn more about the process of involving people in genomic research 

3. develop a standardised way of planning, reporting and evaluating involvement in order to 


improve future genomics research. 


This thesis will explore ways of co-defining problems, co-creating solutions and co-evaluating methods of 
involving people. It will demonstrate practical ways of involving people in different stages of genomics 
research across four different communities of shared interest. The research activity undertaken to 


achieve these aims was as follows. 


1. Review of the literature to assess methods of public involvement that may be applicable for human 
genomics research (or those that are already being used). 

2. Review of current human genomics projects around the world, to understand the degree of 
involvement currently occurring, and the gaps. 

3. Application of participatory action research paradigm to studies of four different groups of people 
associated with human genomics research, either underway or in planning. 

4. Planning and conduct of the involvement activities, and where appropriate, sharing of learning 
experiences and description of how involving people affected or informed future genomic research. 

5. Co-creation of a standardised framework to compare the views and perspectives of different 
stakeholders about genomics research across the four case studies, including views about 


involvement, methods of involvement, and any impacts and outcomes of the involvement. 
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Chapter 3 — Methodological overview 


Methodological summary 


This thesis used multiple paradigms and methods, including mixed methods for data analysis. This 
chapter is therefore divided into sections which explain the background, justification and application of 
each paradigm and method at the various stages of my doctoral research. While it is necessary to 
summarise theoretical perspectives and provide a relevant philosophical discussion of research 
methodology, in this thesis this discussion is embedded in the context of how it was applied and the 
paradigms and methods used, rather than in abstract. An evaluation of the paradigms and methods used 


in this doctoral research can be found in the Discussion chapter, in the section ‘Evaluation of research 


design and limitations’. 


This chapter also outlines how learning from the different stages of research influenced the methodology 
used in the case studies, including research design and data analysis. A chronology of the different stages 
of the doctoral research is summarised in the ‘Thesis timeline’. The timeline shows that this doctoral 
research was guided by an iterative process, and improved by applying the principles of participatory 
research at multiple stages, including during co-design and co-analysis. Qualitative and quantitative data 
were gathered in order to find gaps in knowledge and patterns in data. Learning from these findings was 
used to inform the next stages of the research.®° For example, findings from the narrative and scoping 
reviews influenced my decision to use a participatory action research paradigm to guide the case studies. 
While this chapter outlines the methods used in each case study, to avoid repetition more detail on the 


methods used for each study is contained in other chapters. 


This doctoral research was informed by a ‘rights-based’ paradigm and used the participatory action 
research paradigm to guide the research process for the case studies and for STARDIT. Participatory 
action research (PAR) and related methodologies such as ‘co-design’ are explored further in the next 


section. 


Case study methodology was used to analyse the different projects, and a cross-case analysis examined 
commonalities and variations in the learning from each case study. Other frameworks for reporting and 
assessing public involvement were used to guide data collection, which included the Public Involvement 


Impact Assessment Framework (PiiAF) and the GRIPP2 reporting checklist.”?°” 
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This doctoral research was conducted in a public health research setting. It is intended to be practical in 


nature, demonstrating ways of applying and developing theoretical frameworks and evaluating them. 


This research is underpinned by the World Health Organization’s ‘Declaration of Alma-Ata’ of 1978, which 
states that “the people have the right and duty to participate individually and collectively in the planning 
and implementation of their health care”.7® It is also informed by the United Nations statements on 
Human Genetics and the ‘Report of the IBC on updating its reflection on the Human Genome and Human 


Rights’.°° 
Research paradigms 


Health research requires a pragmatic approach to understanding problems and creating solutions and, 
accordingly, a pragmatic approach has been taken in this thesis. Multiple research paradigms have been 
used to understand and solve the problems being examined. The word ‘paradigm’ has its roots in words 
which mean ‘pattern’ or ‘model’. In plain English, ‘paradigm’ means an ‘approach’ or ‘way of thinking’. 
For example, the terms ‘co-design’, ‘co-production’ and ‘co-evaluation’ describe a set of core values and 
principles applied at different stages of the research cycle. The terms describes multiple methodologies 
with similar components including knowledge and power sharing, involving people in every stage of a 


process, rather describing one single discrete methodology.® 


In research, how a researcher uses a paradigm to understand the nature of knowledge (epistemology), 
ignorance (agnotology), or reality itself can affect why the research is conducted, how, and what it is 
possible to learn from it.®° A pragmatic approach in health research attempts to unite qualitative and 
quantitative approaches, which are often considered as opposites. Quantitative research is associated 
with objective knowledge and positivist perspectives, while qualitative research draws on subjective 
knowledge and an interpretivist perspective. Both attempt to map reality and both have acknowledged 
limitations.®°°2) Researchers in applied fields such as evaluation research have moved towards 
pragmatism as a ‘paradigm of choices’ in research methods, rather than being confined to any particular 
narrow theoretical framework. The pragmatic method simply uses both qualitative and quantitative 


methods, whenever appropriate.5°?57) 


The paradigms used in this thesis — such as participatory health research and critical pedagogy — actively 
dismantle ‘labels’ and linguistic separations between ‘the researcher’, ‘the researched’ and the research 
itself. Such an approach requires careful explanation and evaluation of these paradigms, justification for 


their use, and an account of how they were used. While they can be hard to define precisely, in this thesis 


33 


public statements about ‘paradigms of involvement’ are included as a measurable indicator of 


involvement in STARDIT. 


It is worth noting that while this doctoral research is informed by multiple paradigms, it is being applied 
within the context of genomics research, where there is objective data in the form of the ‘hard’ code of 
DNA, which is itself objective, not subjective. While understanding, interpretation and technology are 
constantly evolving, concepts of power sharing in this highly standardised area of research need to be 
explored in parallel. The creation of a standardised way of discussing involvement was therefore required 
in order to work within the highly standardised processes required for genomics research and analysing 


DNA.” 
Rights-based paradigm 


This doctoral research has been influenced by a number of rights-based paradigms, primarily a human 
rights-based paradigm. While this section will not explore the entire history of human rights, it will 
explore how current codified rights are influencing involvement in genomics research. This doctoral 
research is grounded in such paradigms, seeking to both act within them, uphold and promote the ideals 


codified in these documents. 


Human rights 


The United Nations describes human rights as “inherent to all human beings”. The United Nations (UN) 
1948 Universal Declaration Human Rights states ”all human beings are born free and equal in dignity and 
rights”. Significantly for genomics research and data sharing, it also states all humans should be able to 
"receive and impart information and ideas”.** The UN also states that the concept of democracy is 
inherently interconnected with respect for all human rights and ”fundamental freedoms are 
interdependent and mutually reinforcing”.°* The World Health Organisation’s 1978 ‘Declaration of Alma- 
Ata’ stated ”the people have the right and duty to participate individually and collectively in the planning 
and implementation of their health care”*®, further connecting concepts of democracy and self- 
government with universal rights in healthcare implementation. The UN’s 1997 ‘Universal Declaration on 
the Human Genome and Human Rights’ further codified how genomics interacts with concepts about 
individual and collective rights, and influenced international law and ethical norms in genomics 


research.” 


The United Nations has provided much guidance on working with Indigenous peoples around the world 


% and the 'Declaration on the Rights of Indigenous Peoples United Nations' was a guiding paradigm 
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during this research”, including the statement “Indigenous peoples have the right to be actively involved 
in developing and determining health, housing and other economic and social programmes affecting 
them”. Similarly, both FAIR°® and CARE?” data sharing principles were integrated into STARDIT, in order to 


ensure reporting aligned with international data sharing principles. 


Informed by the United Nations ‘Universal Declaration Human Rights’ statement that all humans should 
be able to ”receive and impart information and ideas”?t, this doctoral research was also influenced by the 
Open Access movement, which can be considered part of this paradigm, in particular for those who 
cannot afford to access genomics research behind a paywall.?° In 2020, the United Nations Secretary- 
General stated ” purposes that involve data and analytics permeate virtually all aspects of our work in 
development, peace and security, humanitarian, and human rights”, encouraging “everyone, 
everywhere” to ’nurture data as a strategic asset for insight, impact and integrity — to better deliver on 
our mandates for people and planet” .?? All articles which are part of this thesis have been submitted for 


publication in peer-reviewed journals which are Open Access, free for anyone anywhere to access. 


Within the paradigm of human rights are the rights of women and children, codified in the UN’s 
‘Convention on the Rights of the Child’ and the UN’s statements on gender equity and equality. 7? The 
principles of self-autonomy and individual choice in health are monitored by the UN, in particular, women 
having the right to decide whether to terminate pregnancies.’ In addition, the United Nations 
Convention on the Rights of Persons with Disabilities is also a guiding paradigm in this thesis. The specific 
implications of these rights paradigms on genomics research, including using data from DNA analysis to 
make informed decisions about parenthood, concepts of body sovereignty and concepts of ‘healthy 
births’ and eugenics are explored in the Discussion section ‘Implications for public discussion debate on 
the concepts of youshéng (eugenics)’. 


Consumer rights or human rights? 


While the connection between human rights and democracy is significant, it is important to note that 
human rights and concepts of ‘social democracy’ can also be contrasted with ‘consumer rights’. From one 
perspective, the social democratic rights-based paradigm relies on collective action to create public 
health initiatives, codified by the World Health Organisation (WHO), which stated that health promotion 
is the process of ”enabling people to increase control over, and to improve their health” .1% Similarly, 
Article 4 of the UN’s Universal Declaration on the Human Genome and Human Rights states ”the human 
genome in its natural state shall not give rise to financial gains”.°? Genomic science will require a 
renegotiation of the social contract of public health systems and involving the public in this process is a 


priority 3°”), This was summarised in the 2016 Annual Report of the UK’s Chief Medical Officer, who 
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wrote, "the importance of ethical reflection and patient engagement in the development of the 
coordinated national and international developments in genomic medicine should be recognised”. 3+(P27) 
With the current and future genomics research and services likely to be a mixture of state and private 
partnerships, the involvement of the public is essential to balance individual rights against the public 
interest.2° Genomics and other biomedical research present specific social, ethical and legal challenges. 
As the communication and commercialisation of research results continues, it is essential to balance 
individual rights against concepts of ‘the collective good’, and data protection. The Nuffield Council on 


Bioethics wrote in 2012: 


”To address these challenges, leading international institutions stress the importance of public 


involvement in biomedical research and innovation” ^42 


Parallel to the human-rights based paradigms (but not independent of them) is the ‘consumer rights’ 
paradigm, where people are involved as ‘consumers’, ‘users’, ‘tax-payers’, ‘payers’ or ‘customers’. This 
model is grounded in free-market paradigms, based on the axiom that the market model will create 
services that are needed in response to the needs of the customers.” The ‘consumer’ paradigm can be 
seen as parallel but not identical to the ‘social democratic’ idea of public involvement. The history of the 
use of the word ‘consumer’ goes back as far as the Latin roots of the word, with roots in a word which 
means ‘under’, ‘below’ or ‘sub’. The origins of the word are associated with a transactional merchant 
relationship where the ‘consumer’ takes goods or services, and to ‘consume’ can mean to use up or 
waste.?° Yet in the last 30 years the word ‘consumer’ has commonly been associated with the ‘consumer 
rights’ movement, in particular in the context of health services and research. A recent assessment of the 
influence of public involvement on health research concluded that a ”consumerist approach is still 
predominant and that in reality the public voice has limited impact upon the research design or upon 


which research gets funded” .1 


While more recently the UK, USA, Canada and other Anglophone countries have moved away from using 
the word ‘consumer’ to describe a member of the public, countries such as Australia continue to use the 
word in Government policy. 1 For example, the term ‘consumer’ is contentious in the UK *°8, and has 
been actively rejected by some, being labelled as an ”overtly consumerist approach to involvement 
associated with internal-market legislation of the 1990s Conservative administration” which is in contrast 
to a citizens' rights based approach. 1 A recent UK report about the ethics of data sharing referred to 
“mere consumers, reduced to accepting or refusing an option presented to them, or choosing between 


m7 


several available ‘offerings’” ’ while the Labour party in England wrote that people should not be “mere 
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consumers of services but genuine and active partners in designing and shaping their care and support”.® 


Similarly, commentators in the United States have argued the term ‘consumer’ even sits below ‘customer’ 


in terms of power relations. 


“It is a mistake to assume that patient involvement in the system is “consumerism” in either of its 
common meanings: 1) acquisition of goods and services in ever-increasing amounts, or 2) The 


protection or promotion of the interests of consumers”. +° 


However, current Australian Government language describes consumers as ‘potential users of healthcare 
services’. While the current 2016 National Health and Medical Research Council ‘Statement on consumer 
and community involvement in health and medical research’ states “Collectively, ‘consumers’ and 


In 


‘community members’ may be referred to as ‘the public’”, the word ‘consumer’ is used by government, 


charities and advocacy organisations as a shorthand for ‘the community’ and ‘the public’ .1°” 


Similarly, DNA tests which people can buy, independent of healthcare systems, are often described as 
‘direct-to-consumer’ (DTC) by genomic researchers and governments, who seek to differentiate them 
from other kinds of DNA testing for either healthcare, research or use in the criminal justice system.*%4 
The question of regulation of these services is central to this thesis. In other words, who decides who 
regulates whom? As one commentator wrote, "there is a sort of a paradox between the lagging 
implementation in health care of the few genetic tests with proven clinical utility, on the one hand, and 


the speedy DTC offer of tests, with or without clinical utility” .2”” 


With people no longer reliant on governments or healthcare providers for access to their own genomic 
data and any subsequent analysis, people have new power as 'consumers'’. Tritter and McCallum argued 
in 2006 that this shift from ” patient to customer, user, or consumer has been central to neo-liberal 
approaches to rationalising health services, facilitated by managerialism and marketization”.* From this 
perspective, true ‘user involvement’ or ‘consumer involvement’ in genomics research is presented as a 
feedback mechanism for consumer views, which is an essential part of markets-driven services. 14° 
However, without real market choice, or the enabling conditions to make an informed decision, people 
must rely solely on giving ‘feedback’ in the hope of influencing service provision. For example, if a for- 
profit DNA testing company informs customers they will sell customer DNA data to those doing genomics 
research (as is indeed the case with both Ancestry DNA and 23andMe™*), customers have limited power 


in this situation, and have options which include not using the service or providing feedback. Accordingly, 


both the social democratic and consumerist paradigms are significant in public health genomics and 
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genomics research. Ensuring that research is “civically responsible”*° requires navigating how the public 


can be involved in both social democratic and consumerist paradigms. 
Individual rights or collective solidarity? 


Concepts of ‘individual rights’ and ‘collective solidarity’ need to be explored when applying rights-based 
paradigms, as the very nature of genomics research transcends the individual in order to understand 


individuals’ variations in the context of whole populations, and sometimes other species. 


A report from the UK’s Human Genetics Commission in 2002 articulated a way of considering data sharing 
in genomics in a fair and ethical way. By acknowledging that individuals who choose to share data may 


indirectly benefit others, the public are empowered to make a balanced and informed decision: 


“nobody should feel pushed into taking part in genetic research, when they make this decision 


people should be aware that by taking part they might help those suffering from disease”. 14(r8) 


The report articulates that genetic knowledge may bring people into a “special moral relationship with 
one another”, and proposes the concept of ” genetic solidarity and altruism, which promotes the common 
good”. 145(P8) Increasing ‘public awareness’ of genomic technology was a key recommendation of a report 
by the Human Genomic Strategy Group in 2012 that also looked beyond national boundaries in solidarity, 
citing how genomics data sharing can be used to "benefit the care of patients across the NHS and indeed 
the world”. 46°”) Similarly, Genomics England stated in 2015 that the comparison of specific rare disease 
sets across the world could add "immeasurably to knowledge” and potentially have ”world markets”. 
117(P11) These concepts are simultaneously being “eroded” by language such as ‘personalised medicine’, 
which ignores the enormous collective data-sets required to allow these interventions to arise. 18 Thus, 
careful attention needs to be given to both the language used to describe genomics research and related 
medical interventions and the ethical arguments in support of involvement and participatory decision 
making. Any participatory action research in this area needs to include an awareness of this language and 


attempt to involve people in mapping preferences and existing power structures. 


A 2015 systematic review of ‘stakeholders’ perspectives on biobank-based genomics research 
emphasized the importance participants of biobanks felt of being informed about data-sharing practices, 
measures for privacy protection and applicable sanctions if the data were misused."’°Participants 
regarded reidentification as a negligible risk as the benefit of data sharing for science and society 
outweighed the potential risk and they were generally very supportive of data sharing. 1° Interestingly, 
the same review suggests that unless there is experience of or evidence of serious data breaches, overly 


rigid data protection could constitute ‘paternalism’ over patient preferences and be described as an 
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“unjustified obstruction to research”. t1? Risks such as perceived data breaches could be reduced by 
involving the public, who can be involved in scrutinising data storage and access, and accessing if it aligns 
with perceived values of what defines a ‘breach’ of trust in data storage and access. Involving the public 
in shaping genomics research can create research which is more likely to benefit more people. 72!°6 
Emerging biotechnology should be subject to ‘public ethics’ and decision making which is orientated by 


the pursuit of the public good.4(°22) 


Issues such as ‘shared ancestry’, and exactly how many generations back one means when one uses the 
term ‘shared ancestry’ are central to all of the case studies in this thesis. People from all of the 
populations | worked with will be affected by decisions made by others in those communities, either in 
communities defined by shared ancestry, cultural or other shared interests. For example, people in either 
the donor conceived group, or the Aboriginal community, will share significantly more DNA variations 
with others in that community than with people from populations such as ones defined by national 
borders. Similarly, potential participants of the ASPREE multi-generational research study and those from 
the ausEE study may discover things about their genome which may have relevance to close relatives, 
including offspring. It is therefore possible that people who have not given consent to participate in 
genomics research may still discover information about how genomic variations they might have will 
affect risk of disease, or that such data obtained from close relatives may be used by the criminal justice 
system. In summary, decisions of close relatives to participate in research can directly affect people who 
have not given consent, and thus concepts of individual consent needed to be re-examined in this 


doctoral research. 


Accordingly, when applying a participatory action research paradigm, the way in which data is shared 
must therefore be co-designed with participants and other stakeholders from the outset of any study 
which collects genomic data, in order to ensure individual and collective rights are balanced, and people 
at greater risk of exploitation are not exploited or exposed to harm. By ensuring those being asked to 
participate are also invited to be involved and supported to make informed decisions, it is more likely the 


research will be ethically acceptable and meet the needs of those participating.?03333-120 


If genomics research does this poorly, and if it makes it difficult or overly complex for people to give 
informed consent to participate in genomics research, those people not participating may mean 
humanity’s understanding of genomics and disease will be poorer, and more people will likely suffer from 


disease. An indirect outcome from improved public involvement in genomics research may be more 
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people volunteering to share their data, and thus more useful data which can be used to help more 


people. 
How the rights-based paradigm shaped this doctoral research 


Concepts such as ‘involvement’, ‘ethics’ and ‘democracy’ cannot always be expressed fully in linguistic 
constructs, as they carry resonances of concepts such as equality, justice and human rights. 72+ Further 
still, translating such descriptions between cultures and languages presents difficulties. For example, in 
the context of research, at the root of the word ‘ethics’ is not one single definition, but an entirely 
subjective concept, expressed in English by the words ‘fairness’ or ‘justice’. 12? Similarly, the concept of 
‘involvement’ in research (and associated concepts) is connected to these concepts and thus there is an 
array of subjective feelings about these concepts, seeking expression in a limited English lexicon. Terms 
such as ‘consumer’ and ‘community member’ also raise questions of ‘who decides who decides who is in 
each of these categories’ or groupings. The need for clarity in such terminology, and ways of working 
across human languages to provide such standardised terminology was an objective of this doctoral 
research. These issues are explored in more detail, along with proposed solutions, in the section 


‘Standardised Data on Initiatives — STARDIT: Beta Version’. 


Participatory action research 


This section will describe the participatory action research paradigm, and how it was used to involve four 
different groups, in varying formats. As the methods used for each group varied, here the rationale for 


selecting various participatory action research methods will be provided. 


Informed by the reviews conducted as part of my doctoral research, | chose a participatory action 
research paradigm to guide the process with co-design and reporting informed by guidance from a 
number of sources.’*°”123 Participatory action research is an umbrella term which describes several 
related approaches, including forms of action research which embrace a participatory philosophy and 
include ‘co-design’ and ‘co-production’ of research. ** These approaches share a process whereby 
researchers, the public and other relevant stakeholders “work together, sharing power and responsibility 


from the start to the end of the project”,®° including knowledge generation and translation.®° 


An important contextual consideration in participatory action research is the concept of individuality, and 
the extensive laws, rights and moral codes which enshrine the protection of ‘self-rule’, with the 
biomedical model subordinate to the rights of those considered competent to make informed decisions. 


124(P58) However, ‘self-rule’ or autonomy is only part of the context in which genomics research operates, 
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and does not fully articulate concepts such as the trust one may place on experts to make decisions on 


our behalf 159, in what might be described as consensual heteronomy. 


As discussed in the human rights paradigm section, the tension between ‘private interest’ and the ideal of 
‘public good’ must be explored as part of the participatory action research process. Concepts such as 
‘public good’ in the context of genomics research cannot be defined by an empirical collecting and 
collating of multiple self-interests (such as public surveys), so participatory action research provides a 
guiding paradigm to help navigate the complex ethical issues which must be fully explored in order to 
balance and prioritise the sometimes competing rights of individuals. 175°?) In other words, something 
other than individual preference is required to explore value conflicts. 5°19 How this is actualised in 
participatory action research and in the emerging and complex field of genomics research is what is being 


investigated as part of this doctoral research. 


Guided by the participatory action research paradigm, | worked in partnership with a number of 
organisations and communities in order to involve people in discussions about the future of genomic 
research. Where possible, potential participants were included in the design of each project. The main 
groups | interacted with, and the results of involving them in the research process, are briefly introduced 


below and will be presented as case studies in the format of their submission for publication. 
Defining participatory action research 


Participatory research is an umbrella term which describes a number of related approaches, including 
84(p1). 

community-based participatory research 

participatory action research (including critical participatory action research) 

participatory health research 

community-partnered participatory research 


cooperative inquiry 
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other forms of action research embracing a participatory philosophy which may include ‘co- 


design’, ‘co-production’ and ‘co-evaluation’ of research. 


The term ‘participatory action research’ will be used to refer to all variations of this approach, unless 
explicitly stated. At the core of participatory action research is critical reflexivity, a process which asks 
people involved to reflect on the causes of problems, any solutions and the actions that people can take 


to improve the current situation. 176° It is a form of collective self-reflective enquiry undertaken by 
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participants in order to understand their situation from a number of perspectives, including rationality 


and a sense of justice. 127153) 


In a health context, participatory action research attempts to reduce health inequalities by supporting 
people to be involved in data collection, reflection and, ultimately, actions to improve their own health 
228 It is an interactive process, seeking to understand and improve things through change. 77° 
Participatory action research integrates knowledge translation into the research process, by involving 


those who can inform future actions as partners in the research. 


While participatory action research can be a challenge to define, the quotation below from Orlando Fals 
Borda summarises both the underlying philosophy and ways of implementing the participatory action 


research paradigm, including participatory knowledge translation?”*: 


Do not monopolise your knowledge nor impose arrogantly your techniques, but respect and 
combine your skills with the knowledge of the researched or grassroots communities, taking 
them as full partners and co-researchers. Do not trust elitist versions of history and science which 
respond to dominant interests, but be receptive to counter-narratives and try to recapture them. 
Do not depend solely on your culture to interpret facts, but recover local values, traits, beliefs, 
and arts for action by and with the research organisations. Do not impose your own ponderous 
scientific style for communicating results, but diffuse and share what you have learned together 
with the people, in a manner that is wholly understandable and even literary and pleasant, for 


science should not be necessarily a mystery nor a monopoly of experts and intellectuals. 


The concept of ‘dominant interests’ is especially important in the context of research with Indigenous 
peoples around the world and the UN’s recognition that their culture can be threatened by ‘dominant’ 
cultures**, which was explored in the previous section ‘Human rights’. Methods of mapping such 
‘interests’ in a standardised way is discussed further in this chapter in the section ‘Standardised Data on 


Initiatives — STARDIT: Beta Version’. 
Defining who is participating 


Defining the people and communities involved in participatory action research can be difficult. Using a 
definition whereby anyone who is not a professional researcher is a member of the community can blur 


lines which are already ill-defined. °°) 
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Other traditional labels can be unhelpful in the participatory model as the labels project power structures 
onto people. For example, ‘service user’ may include anyone affected directly or indirectly by a service, 
‘consumer’ uses an economic market model to classify health and research, ‘patient’ places it within a 


medical model and ‘survivor’ provides are more psychiatric perspective of those ‘affected’. 37) 


While co-creation of a term that people would like to use to describe themselves would be best-practice, 


practically a definition must be arrived at from the outset of the research. 
Origins and parallels 


The origins of participatory action research have roots in methodological critiques of conventional 
research, including positivist research, which did not acknowledge power structures inherent in research 
and other settings. In particular, this kind of research did not acknowledge the position of relative power 
of the researcher in comparison to the research participant.‘ At the core of participatory action research 


is Paulo Friere’s concept that reflection about the world requires action in order to transform it*?2(°): 


“to see the world not as a Static reality, but as a reality in process, in transformation”. 


In this paradigm, reflection and action are interconnected in ‘conscientisation’, where critical thinking is 
developed through dialogue and involvement.? This approach is also connected to the concept of 
‘critical pedagogy’. Freire summarised this by stating that ”reflection without action is sheer verbalism” 


and “action without reflection is pure activism, or action for action’s sake”.?7° 


This perspective requires viewing the world as something changing and changeable. This was articulated 
in a 1993 World Health Organisation report which articulates alternative futures, dividing them into 
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possible, plausible, probable and preferable. 135) See Figure 3.1 for a visualisation of this concept. 
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Figure 3.1: Alternative futures 
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Such a view of preferable alternative futures immediately raises the question ‘preferable for whom’, 
challenging health researchers and policy makers to balance the varying and sometimes competing or 
conflicting needs and preferences of multiple stakeholders. Participatory action research offers an 
opportunity for multiple stakeholders to be involved, including people affected by certain health 
inequalities to take part in shaping this future, collectively attempting to articulate and then achieve what 
is preferable. This process involves learning from the past, examining potential realities, likely realities 
and preferred realities through an interactive process of involving communities of shared interest in 


appraising, shaping and affecting these potential realities. 14°24) 


Participatory approaches in research share a number of significant connections to existing qualitative 
methodologies and methods, yet the constructivism at the core of participatory action research ensures 
that this research method reflects that there is no ‘one truth’ or objective answer, but that there are 
multiple ways to ask a question and try to improve health inequalities. °°?) Involving people in health 


and research in this way recognises that some improvements in health can only be achieved by actively 
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involving people. 175°") Effective public health, and thus, public health genomics, requires a range of 
methods beyond epidemiological methods in order to reflect the diversity of the issues it is trying to solve 
135(p174) The question of which or whose values should direct decisions in health and research, while 
complex, can be navigated by participatory methods, as long as they are transparent, accountable and 


with clear boundaries for decision making agreed in advance. 16115) 


Epistemology is the study of what is known and how we know it is, yet a more recent but equally 
important discipline is ‘agnotology’, which is the study of what we don’t know, and why.?* This includes 
crucial concepts such as culturally induced ignorance, which are central to applied critical pedagogy and 


the participatory action research paradigm. 
Contextual considerations 


There are a number of interrelated areas which require explanation in order to give participatory action 
research a meaningful context in relation to this thesis. The following sections explore these important 


perspectives. 
Positivism and scientism 


The positivist forms of knowledge which can be applied to social sciences (and other natural sciences) 
require the researcher to aspire towards an objectivity from that which is being researched. At the core 
of this thinking is a reductionist philosophy that if all component parts of a system (including society) are 
understood, reliable predictions can be made. Critics of this reject this as ‘scientism’, which is the belief 
that true knowledge can only be gained using empiricism. 1°”°29) While this system can be helpful in some 
contexts and has led to many scientific advances, it reaches difficulties when trying to articulate and 


codify concepts such as values, emotions, lived experience of disease and ethics. 


Positivism often requires people aspiring towards objectivity to then interpret, codify and analyse both 
quantitative and qualitative data. This often includes attempting to collect and analyse data in relation to 
the subjective aspects of people’s lives, such as ethical values. It also requires subjective judgement on 
the part of the researcher in relation to research design (methods, analysis and reporting) and requires a 
consideration of how best to aspire towards objectivity. 1615 Participatory action research differs from 
more traditional forms of research as the conceptual division between the researcher and the researched 
is blurred, removing the ‘object’ from research, making all stakeholders partners in the process. By 
removing ‘subjects’ or ‘respondents’, the reductive methods of more traditional kinds of research are 
avoided, with the ‘data’ remaining in context as a record of subjective experience. The participatory 


action research paradigm accepts that humans cannot separate conscious experience from the concept of 
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external reality in order to describe an object. Human experience is where subject and object meet, with 
any subsequent resulting action affecting future experiences of subject and object. ?78The study of 
genomics adds yet another layer of complexity to this, with our individual DNA code (which exists as an 
objective object) in some cases affecting our individual instances of subjective, conscious experience 
(qualia). For example, our DNA can affect our colour perception, with some people being labelled as 
‘colour blind’. Subsequent interactions of our inherited DNA with our environment (including culture), or 
‘nature or nuture’ questions, require careful articulation of who is deciding what is described as 


subjective or objective, and why. 


The aspiration towards objectivity and detachment when applying such positivist paradigms is particularly 
difficult for exploring people’s views and preferences about genomics research, as no one person is 
detached from being affected by this research, with genomics research having a quality of ‘solidarity’ or 
‘shared-ownership’.1*5(°5) In contrast, participatory action research requires no separation between 
subject and object, recognising that the two are combined in the experience of the individual, and 
enhanced by communication with others. A person’s experience of group dynamics, inter-personal 
relationships and a sense of community all affect the relational knowledge of the individual. 12615 This 
kind of critical participatory action research requires a ‘self-critical’ perspective, as the research process is 
not simply one of collecting data, but attempting to nurture self-reflection, recognising that the 
experiences of the researchers and participants are interwoven and must be acknowledged. This is 


particularly true for research involving autoethnographic methods, which is explored further later in this 


chapter in the section ‘Research with family members and close relatives’. 


Freire’s influence on the participatory action research paradigm can also be applied to qualitative 
research, challenging 'interpretivist' methods, which place confidence in the value of subjective accounts 
of lived experience as being authentic and reliable.*?® Freire’s epistemology rejects that consciousness 
creates a reliable and infallible copy of reality. Participatory action research would recognise that 
accounts of lived experience are constructed from certain power dynamics and cultural traditions. Such 
dynamics themselves require critical appraisal, including asking all stakeholders to actively reflect on 


power relations, and where appropriate, challenging or improving them. 1381154) 


The combined relational and reflective knowledge which is created by the participatory action research 
process is an ‘embodied knowledge’ which creates an important link between knowledge and action, with 
the translation stage of the research cycle effectively taking place within the individual. 16°19 By 


extension, this research method also transcends the research process and ‘the researched’ people and 
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can extend into changed individual behaviours, thus into communities and the wider public. 
Methodological theories explaining the importance of measuring such transformations and impacts are 


explored in the section ‘Theoretical approaches to case studies’. 
Democracy 


In 1969, Arnstein wrote that the ‘participation of the governed in their government is, in theory, the 
corner-stone of democracy’?’. Fewer than ten years later this sentiment was enshrined within the World 
Health Organisation’s ‘Declaration of Alma-Ata’ which stated ‘the people have the right and duty to 
participate individually and collectively in the planning and implementation of their health care.’*® This 


was followed up by the following WHO statement in 19867°: 


Health promotion works through concrete and effective community action in setting priorities, 
making decisions, planning strategies and implementing them to achieve better health. At the 
heart of this process is the empowerment of communities - their ownership and control of their 
own endeavours and destinies. Community development draws on existing human and material 
resources in the community to enhance self-help and social support, and to develop flexible 
systems for strengthening public participation in and direction of health matters. This requires full 
and continuous access to information, learning opportunities for health, as well as funding 


support. 


Participatory action research is highly dependant on the social and political context, with democracy 


130(P5) With complex (and often international) communities 


regarded as an important enabling condition. 
evolving online, an important emergent enabling condition is the free exchange of information regardless 
of borders®', which in practical terms means access to technology and an uncensored internet 
connection. These democratic principles, when applied to internet access mean that the location of the 
person involved is often secondary in consideration to their ability to connect to communities of interest 
uninhibited by technological restrictions or censorship. For example, traditional ideas of ‘citizenship’ and 
nationality are less important to people affected by rarer diseases caused by a genomic variant than 
being able to connect online and form communities of interest with similar people, regardless of location. 
As Steven Pinker wrote, ” people who have grown up with the democratization of information will not 
tolerate paternalistic regulations that keep them from their own genomes, and early adopters will 
explore how this new information can best be used to manage our health”.1*? People affected by rare 
diseases are already creating online communities which span continents and involve those people 
affected in helping decide research priorities.°1** As participatory action research can be international, 


with people sharing local and national knowledge at an international level*2"), this paradigm can guide 
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how such communities interact with more traditional research stakeholders such as government and 
industry, with democracy as an underlying principle guiding how participatory action research is used. 


However, as George Orwell noted’: 


In the case of a word like democracy, not only is there no agreed upon definition but the attempt 
to make one is resisted from all sides ... the defenders of any kind of regime claim that it is a 
democracy and fear that they might have to stop using the word if it were tied down to any one 


meaning. 


Thus the task of constantly co-defining what democracy means, what it looks like, and how it works in 
relation to genomics research is an ongoing task, with participatory action research at the core of how 


the meaning of such concepts can be collectively agreed and acted upon. 
Research with family members and close relatives 


Participatory action research is one of the ways of over-coming potential ethical issues of close family 
research by attempting to address power imbalances by making sure everyone is involved in decision 
making at every stage of the research. The decision to work with my own family members (including my 
own mother) as described in the Shared Ancestry case study was a decision made after consulting with 


both methodological and ethical advisors. 


As the concept of ‘researcher objectivity’ is challenged by participatory action research, having a 
researcher involved who ‘has a stake’ in the issue or is an ‘insider’ and is also affected may improve trust 
in the process among those participating. ‘43 In addition, as a researcher with an ‘insider’ status, | was 
more likely to be able to offer a novel interpretation of the data than someone ‘outside’ or unconnected 
to the research. While having the perspective of being an ‘insider’ or a person comparably affected can 
aid a researcher in understanding and empathising with the other research participants, the status of 
being an ‘insider’ or ‘outsider’ is often blurred.*** Similarly, participants might view themselves as 
‘outsiders’ of the research system, able to offer creative input or challenge researchers’ assumptions.*“* 
Participatory action research also challenges the binary dichotomy of ‘insider’ or ‘outsider’ status, which 
is particularly relevant to genomics research in relation to concepts such as ‘shared ancestry’, where 
there is no definitive articulation of when ‘shared ancestry’ begins or ends, and thus must be co-created 
with participants.?* In other words, all life on earth has ‘shared ancestry’, so where exactly do we draw 


the linguistic line, and who is the ‘we’ that gets to decide? 
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As some of the data analysis in this thesis involved analysing data from my own close relatives, or people | 
am closely related to, the study was both ethically and methodologically complex. Having a researcher 
involved in the project who is considered an ‘insider’ by research participants may also improve trust in 
the process among those participating.““? Informed by autoethnographic methods, | took a number of 
steps to improve both the quality of the data collected and the analysis.’ As well as keeping a reflexive 
research diary, study team members and research participants were involved in reviewing the analysis 


and commenting, as a way to increase the number of subjective interpretations of the data.’4” 


The requirement of transparency in the research process helps demonstrate a practical application of the 
theories of Karl Popper, regarding the method of science as ‘critical rationalism’, which rejects empiricism 
and ‘observationalist-inductivist’ forms of knowledge, instead, asking, ‘does what is observed align with 
other observations’, and thus can a theory be ‘falsified’ or not.’*® Thus, there is no one single ‘objective’ 
perspective to provide ‘the truth’. For example, even the analysis of data by researchers is affected by 


their own lived experiences. 


The social constructivist methods used in the Shared Ancestry case study here provided a way of 
exploring and using this subjectivity to deepen understanding, incorporating the valuable data this 
subjective perspective provides. This perspective aspires towards a shared ‘truth’ rather than an abstract 
concept of ‘objectivity’ which an ‘outsider’ has, and an ‘insider’ does not. For example, the process of 
analysing online discussion is complex and involving research participants in analysing the data is an 


‘interpretive’ process, relying on a social constructivist approach. 


Popper wrote?: 


“the search for truth is only possible if we speak clearly and simply and avoid unnecessary 


technicalities and complications”. 
Using STARDIT (including standardised preference mapping) introduced some clarity and consistency into 


the complex case study, providing a standard framework for data analysis, and thus facilitating 


comparison with other case studies. 
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The process of participatory action research 


As the name suggests, participatory action research requires an action, something which is changed and 
then evaluated collectively. 127153)131136) A number of practical tools exist to support researchers starting 
a participatory action research process, enabling them to assess whether the research can be described 
as ‘participatory’. For example, checklists exist which ask questions such as ‘is the community of interest 
clearly described or defined?’.*°° Guidance suggests that participatory action research must be located in 


the reality of daily life. 17° 


t9 For example, if people are members of online communities defined by a 
shared interest in a rare disease, the research must take place inside these structures, in an accessible 
way. Participatory action research can be carried out by various stakeholders including members of the 
public, health professionals and academic researchers, with all members of the group having equal 
influence or “equitable co-governing powers”. 12619.1518) The following sections explore the stages of 


participatory action research considered in the case studies described in this thesis. 
1: The purpose and appropriateness of participatory action research 


The purpose of participatory action research is to help assess what needs to be done, take action and 
then evaluate any actions that take place. It aims to help participants to improve their understanding of 
problems and to help elucidate any potential actions they can take. 11135) Participatory action research is 
an appropriate method when an issue or problem is complex or may involve a number of ethical issues 
which require resolution. If conducted correctly, it should help participants understand their own 


circumstance and be able to make an informed choice about any future actions. 15100138) 
2: Design of participatory action research 


It is considered best-practice to include participatory action research elements in the research design. 
130(P2) A review of participatory action research suggests the method can improve how culturally 
appropriate the research is to the group involved and also ensures the method is logistically realistic. 
151(P8) We involved potential participants of all three case studies where we conducted research in giving 
feedback on both the research design itself, and also how we planned to involve participants in the 


research process. 


With participatory action research there are two kinds of reflection that are relevant when designing 
research. Firstly, personal reflection, which examines the personal assumptions, values and experiences 


of all stakeholders involved. Secondly is epistemological reflection, which requires a recognition of the 
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limits of research methods, and ultimately, science itself. 1°14 Such exploration of both the strengths 
and limitations can help ensure that all stakeholders have realistic expectations about what can be 
achieved by incorporating participatory action research into research design. Multiple stakeholders were 


involved in this process for all of the case studies described in this thesis. 
3: Ethical considerations before starting participatory action research 


In addition to all the ethical considerations of any kind of research, the participatory action research 
process has additional ethical issues to consider. Researchers must be cautious of raising expectations at 
the start of the research process that might not be realistic. For example, making it explicit whether the 
process will examine the current context and potential actions, or actually carry out an action and 
evaluate it. 11143) An exploration of attitudes about confidentiality must also take place at the start of 
the research to ensure that incorrect assumptions have not been made. For example, it may be incorrect 
to assume participants wish to remain anonymous, as they may wish to be co-authors. Different 
participants might have different preferences so these must be balanced with the research design and 
agreed in advance. 11144 By exploring such attitudes during the co-design stage of the case studies, we 


hoped to ensure no such incorrect assumptions had been made. 


Another ethical consideration is the potential for conflict arising during the participatory action research 
process, stemming from differing levels of perceived status and power between stakeholders. 126122) 
Consideration also needs to be given to those who may be intentionally or unintentionally excluded from 
the research. 28) While this will always remain an unknown to some extent, certain actions can be 
taken to reduce exclusion and create the enabling conditions for people to be involved, for example, 


involving people in the design of recruitment strategies.1°? 


Similarly, certain skills and knowledge are required for some people to participate, for example, using 
online tools to communicate. Skills and knowledge, such as an understanding of the research process and 
working as part of a group, can often be improved through learning interventions.°>°) If there is no 
shared language between communities of shared interest, such linguistic competencies might be 
considered an unrealistic barrier to overcome in some contexts, although in some cases this presents an 
opportunity to partner with certain communities defined by a shared language, co-creating learning 
resources and paying members of the community to be involved in translation of other relevant 


information. 


When representing findings, it is important that the research process preserves the multiple perspectives 


and voices found within the data.13>"!) With the online discussions used in two of the case studies in this 
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thesis, this meant careful documentation of online discussions (anonymising where appropriate) in order 
to directly quote participants, rather than generalise or paraphrase. Accordingly, where possible, we 
incorporated as many direct quotations as possible into the published case studies. As there is still a risk 
that the overall narrative might not represent what people involved in the research felt occurred, we 
invited everyone involved to review the data analysis and write up, in order to ensure they felt the 


findings represented what was said. 


Ethics approval was obtained from respective organisations before starting the research processes for 


each case study. This process is described in more detail in each case study. 
4: During participatory action research 


During the process, everyone involved in the research must reflect on how a number of different factors 
may enable or hinder the research process. +7!) These include personal values or experiences and 
potentially difficult or negative reactions. For example, might some discussions ‘trigger’ difficult 
emotions? Relationships within the group also need to be considered, including ensuring participants feel 
safe in participating in the research study. We co-created group agreements in both of the online 
discussions in order to ensure participants had an opportunity to shape decisions on any behavioural 
boundaries during discussions. 

For people involved in participatory action research who are not health professionals, the process also 
involves critical health literacy. This is distinct from functional or interactive health literacy as it describes 
how people can act together with others to improve any factors which affect the health of the group as a 


126(P12) As a result, initial questions in the group discussion of the communities of interest 


whole. 
acknowledged power structures, such as universities involved in the research. Discussions then 
attempted to map who currently has the power why they do and people’s views about how it might 
impact on research. For example, such discussions were used to guide the research design for the Shared 
Ancestry case study, ensuring that potential participants were happy with the choice of the facilitator 
who would be used to facilitate the discussions. 

Facilitation is a vital role in the participatory action research process and the person who is an 
intermediary is an egalitarian, attempting to promote equal dialogue, while aspiring to achieve specific 
goals or outcomes.?2©°?) Involving impartial or neutral facilitators for some discussions may reduce 
undue influence on discussions, as leadership requires facilitating shared-decision making at every stage 
of the research. Facilitation needs to create the enabling conditions for good communication, including 
respecting everyone’s individual dignity and privacy. This ensures that people feel they can trust the 


people involved and the process. °°) The non-professional researchers involved in participatory action 


research may perceive the research and the process differently as the research progresses. Anxiety or 
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distrust may change to a feeling of self-confidence and of belonging, if the enabling conditions are 


created, 13%(P13) 


As facilitation is also a process that requires self-critical reflection, we created a private online space for 
the facilitators from both case studies which used online discussion, in order to share experiences and 


reflections, and offer support to one another. 
5: Participatory data analysis — no consensus required 


The participatory action research process does not require consensus, it is the process of uncovering and 
examining different perspectives. Knowledge is created in the communicative spaces, created and 
facilitated by the participatory action research process. The concept of productive conflicts followed by 
useful negotiation is helpful, as it assumes there will not be homogenous perspectives yet also presumes 
the process for managing conflict will be sufficiently robust to result in useful negotiated outcomes 11(P2), 
Involving participants in co-creating boundaries and agreeing upon processes for such discussion and 
ensures that the discussion is aligned with their own expectations and values. If the process is well 
designed, it will engender trust through encouraging shared behaviours which are aligned with the 


universal human values of dignity and respect. 22616) 


This process also allows an exploration of what is known, what we know is unknown, agnotological 
exploration (a study of culturally induced ignorance) and an agnoiological exploration (that of which we 
will always be ignorant). By exploring the limits of both the positivist method and our own knowledge, 
this mapping can produce a helpful framework within which to focus discussion and action. With 
everyone in the role of ‘co-researcher’, collective learning transforms into a process whereby people can 
act based on research findings and have an impact beyond a traditional definition of the scientific 


126 


community. 12°27) In this sense ‘co-researchers’ can then move from a stage of co-designing to co- 


implementing solutions. +? 


6: Sharing findings from participatory action research 


The process of articulating and sharing findings should be participatory, with issues such as authorship 
discussed transparently at the start of the research. 1+1 Accordingly, revisions to the ethics application 
were made in order to incorporate feedback from potential participants as part of the co-design process. 
This included improving plain English summaries of how data from the planned research would be 
analysed, shared, and how participants could be involved in making sure it aligned with their 
expectations. Crucially, the process of involving participants in checking the information being shared 


provided an extra stage of reassurance that no data was being shared that participants did not want 
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shared. This was of particular importance to participants from the ausEE case study who might have 
insurance policies affected by any data breaches, or participants from the Shared Ancestry study who 


could be inadvertently identified in the public domain. 
7: Defining and measuring impact in participatory action research 


Defining and measuring impact from involving people in research is challenging, as outcomes can be both 
short term and long term, and the tasks people were involved in can vary from person to person. 226(P?7) A 
recent review of participatory action research suggested that positive outcomes of participatory action 
research can include ensuring culturally and logistically appropriate research, enhance recruitment 
capacity, generate professional capacity and competence in stakeholder groups, increase the quality of 
outputs and outcomes over time, increase the sustainability of project goals beyond funded time frames 
and during gaps in external funding, and create system changes and new unanticipated projects and 


1512) The examples of negative outcomes identified by the review illustrated why these 


activities. 
outcomes were not a guaranteed product of participatory action research, but rather were contingent on 
key aspects of context. The section ‘Theoretical approaches to case studies’ explores this in more detail, 
including why we attempted to co-define and measure impacts from the participatory action research 


process. 
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Research methods 


Reviews 


As part of this doctoral research, | conducted several reviews and published a summary of my research 
into various systematic review methods in the WikiJournal of Medicine under the title, ‘What are 
systematic reviews?’.’° The methods used for these reviews are summarised in Table 3.1. Further detail 


about each review, the method used and any learnings, can be found in the section ‘Involvement in 


genomics: Mapping what is known and unknown’. 


Table 3.1: Summary of review methods 


Review title Method description 
How are the public This ‘systematic narrative review’ summarised five systematic reviews 
involved in health identified in the area of public involvement in research. Data extracted 


research and what are from the reviews included the kind of involvement taking place (the 


the impacts? A tasks), how it was done (the method), at which stage of the research 

narrative review. cycle it occurred, and any impact on the research that might have 
occurred. 

Public involvement in Using a list of human genomics research projects from an existing 

global genomics database hosted by the Global Alliance for Genomics and Health, | 

research. conducted a scoping review (assisted by co-authors) in which | 


systematically searched public domain websites for information 


reported on involving the public in this research. 


Guidance for planning, A multidisciplinary scoping review, the purpose of which was to show 
reporting and the considerable variation in guidance for reporting, planning and 


evaluating initiatives. evaluating initiatives. 


55 


Case studies 


This section summarises how the case study method was used to describe the four case studies in this 
thesis, including a cross-case analysis. It summarises the literature on case study theories and 
methodology, examines types of case study, and discusses how to ensure quality and rigour in case study 
method. It describes the principles underpinning the choice of the case study methods used in this 
doctoral research and, for each of the four case studies, details variations in the methods used. This 
section also explores case study selection criteria and summarises rejected case studies. The data sources 


for each project are described, including scope, exclusions and methods of analysis. 


Case study methodologies 


Introduction 


Case studies can be used to describe or explain phenomena and the context in which they occur.1* Case 
study methods are used in many disciplines, including anthropology, biography and ethnography. *“° 
Ethnography is a way of recording human events that combines deliberate observation, interpretation 
and analysis.” Case studies can be designed to suit each individual case and research question, allowing 


more flexibility than approaches such as grounded theory.*491°6 
What is the use and purpose of case studies? 


Case studies are useful for generalisation or to challenge generalisations, that is, the purpose of a case 
study is not always to produce outcomes that are generalisable.1 Studying exemplary cases increases 
understanding in any discipline.°’ While it is possible to generalise from one case, consideration needs to 
be given to the power of the example. If it is a ‘black swan’ — an exceptional case that challenges a whole 
theory (for example, ‘there are no white swans’) — a single case can have significant importance when 
generalising beyond the single case.1 Using empirical falsification, it is possible to identify potential 
‘black swans’ and develop a case study which falsifies the proposition that white swans do not exist. 158166) 
However, falsification is limited as a method as it is not always practical or possible to provide evidence 


158(P66) Strategic selection of case studies may be required in order to advance 


that some things are false. 
knowledge in a certain area. Case studies are, therefore, helpful for context-dependant knowledge, 


rather than for creating predictive theories or universal generalisations. 7 
Types of case study 


The diversity of case study methods is reflected in the multiple ways they are described and categorised. 
Conducting a case study employs multiple methods and theoretical frameworks that attempt to capture 


the complexity of the case being explored.14%15%65) The explorative and, often, interdisciplinary nature of 
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case studies requires creativity and innovation, and this means that the multiple methods will constantly 


evolve."49 


In 1995 Stake defined three types of case study: ‘intrinsic’, ‘instrumental’ and ‘collective’. The 
evolution in case study method since then is indicated by the addition of several subcategories of case 
studies, defined more recently by Hyett in 2014. Hyett’s types include ‘illustrative’, ‘exploratory’, 
‘cumulative’, ‘critical instance’, ‘explanatory’, and ‘reflective practice’.1*? These case study types are 


summarised in Table 3.2. 
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Table 3.2: Case study types 


Case study type 


Summary 


Intrinsic 


An intrinsic case study can be used to explore the unique phenomenon of a case as distinct 
from others. Some cases are chosen as case studies because the researcher recognises that 


they have an intrinsic interest that might not be generalisable. 15%?) 


Instrumental 


An instrumental case study uses a particular case to explore an issue or phenomenon in more 
detail by examining the wider context in greater depth. An unusual case often works better 
than one regarded as typical as it is more likely to provide novel data. The purpose of an 
instrumental study is to understand the particular case, but it can also attempt to provide data 
that could produce a valid modification of a generalisation by using inferences.1°2'°10 This 
method requires knowledge of the individual case which can then be applied to knowledge of 


others. 


Collective 


A collective case study involves exploring multiple cases in a sequence or at the same time. It 


provides more context than is possible for one case alone. 


Illustrative 


An illustrative case study provides an opportunity to elucidate a particular area of enquiry and 


seeks to provide a common language about that area. 


Exploratory 


Exploratory case studies can be used to help identify questions and outcome measures prior 


to a larger investigation. 


Cumulative 


Cumulative case studies attempt to combine data from several cases at different points in 


time. They provide an opportunity to generalise without needing to repeat studies. 


Critical instance 


A critical instance study examines a case which has unique interest, often with no intention of 


generalising any findings. It is a useful method for exploring causal relationships. 


Explanatory 


Explanatory case studies can use both qualitative and quantitative research methods to 
explore and describe phenomena, explain causal relationships and develop theory.?© Findings 


from explanatory case studies can be compared for cross-case analyses.16t 


Reflective 


practice 


A reflective practice case study (or ‘reflexive case study’ ) is a post-event analysis that reports 
experiences, insights, debates and discussion.*© It can be used to inform others who are 
encountering similar situations. The method and structure of reflective case studies is 


informed by the ‘reflection cycle’.1% 


Ensuring quality 


As seeking methodological guidance is advised when developing case study methods and theoretical 


frameworks, several experts were consulted as part of the research design for this thesis. 14° 
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In general, a high-quality case study must clearly describe the study design and justify methodological 
decisions.**? However, in a 2013 critical review, Hyett identified that few case studies described the study 


design in sufficient detail.“ 


Improving the scientific rigour of case studies can be achieved by:1491%4 


1. using theoretical sampling (using certain frameworks for certain types of analysis) 
2. asking participants to validate data (involving them in checking analysis) 
3. ensuring transparency throughout the process (describing steps in detail and justifying them) 


4. reporting details of researchers’ relationships with cases, and how these might influence the 


research process. 


A well-conducted case study does not contain intrinsic bias. In fact, rigorous case study methodology 
usually ensures the falsification of more preconceived concepts than other methods available to 


researchers.” 


Theoretical approaches to case studies 


While both qualitative and quantitative data are both valid in case study methods,” in this thesis some 
epistemological approaches regarding how and why data were gathered for the case studies require 


rigorous justification. 


A range of theoretical approaches are summarised in Table 3.3.14°4 Some methods overlap, but this is 


not shown in the table. However, where possible, related approaches are shown adjacent to each other. 


Owing to the often imprecise language used to describe these theoretical approaches, | have used the 
structured data Wikidata categories, which are associated with each approach. Wikidata is used in 


STARDIT reports. In some cases, | have created, amended or improved the Wikidata item.” 


Table 3.3: Summary of theoretical approaches 


Theoretical Wikidata 
Summary 

approach entry 

Interpretive e Attempts to understand meaning from different perspectives. Q2779065 


2 For transparency, | have used my Wikimedia username ‘JackNunn’ for all such edits. The Wikidata categories have 
been formatted as hyperlinks to the Wikidata item, as they are not in themselves references but links to structured 
data. In some cases, | have edited the Wikidata online entry. 
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Theoretical 


approach 


Summary 


Wikidata 


entry 


Attempts to build theories based on these different perspectives. 


Positivism 


Establishes variables in advance. 


Attempts to spot if they fit with an expected pattern or theory. 


Refines and tests the theory in relation to data. 


Limitation: Meets difficulties when trying to articulate and codify 
concepts such as values, emotions or lived experience of disease and 


ethics. 


Q131015 


Critical 


e Questions any assumptions the researcher might have. 
e Includes context such as political or social context. 
e Interprets how power and control potentially affect behaviour. 


Limitation: Focusing only on power relations may mean other factors 


are not fully explored. 


Q301751 


Post-positivism 


e Holds that a researcher’s knowledge, experience, values, theories 
and hypotheses may influence what is observed and how it is 


observed.*® 


e Considers both quantitative and qualitive data are useful, if critically 
appraised for bias. 


e Attempts to build theory from case studies.1%(°39) 


Q2371887 


Constructivist 


Considers that: 


e inthe context of learning (or ‘education’), learners construct new 
understandings and knowledge by integrating them with existing 


knowledge and experience. 


e a well-run discussion facilitated by a neutral facilitator (using a 
number of methods and modes) is central to exploring knowledge 


and values and co-creating new knowledge or preferences. 


Cognitive 


constructivists 


e Views the process of knowledge or experience as being influenced 
by social transaction — as something shared, which allows 
participants to actively reorganise elements of their knowledge (for 


example, as part of an online interaction). 


Q207103 


Social 


constructivism 


e Views the data in a way in which meaning is negotiated through the 


co-construction of knowledge in the discourse. 


Q1135710 
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Theoretical Wikidata 
Summary 
approach entry 


e The researcher has personal interaction with the case.1©° 


Online research e Describes multiple ways that researchers can collect data via the Q6035951 
methods (ORMs) internet, including online focus groups, online interviews, online 


qualitative research and cyber-ethnography (explained below). 


Online- e Adapts ethnographic methods to the study of the communities and Q1816310 
ethnography cultures created through computer-mediated social interactions. 

(or cyber- 

ethnography) 


Theoretical approach used for case studies 


Using more than one theoretical approach is preferable in health research.*** | adopted a combination of 
critical, interpretive and post-positivist approaches to analyse the case studies in order to provide richer 
and more diverse data than using any single method alone. Accordingly, | used two theoretical 
approaches when planning and conducting the case studies: social constructivist and post-positivist. 


These are summarised and appraised in more detail in this section. 


The post-positivist approach outlined by Eisenhardt is an attempt to build theory from case studies. 185(P533) 
In recognition of how a researcher may affect the case study (the observer paradox), | worked with case 
study teams and participants to develop clear protocols aimed at reducing observer paradox, that is, how 
a researcher may affect the case study. This post-positivist approach involved triangulation of multiple 
data sources from both the study team and participants, who were also involved in analysing and 
comparing data.“ To measure the outcomes of involvement, we included in the data collection 
researchers’ subjective accounts of what they learned throughout the process of involving people as this 


represents a valid way of measuring outcomes from involvement.*° 


Combining the methods described above into a single case study method bridged the theoretical 
paradigms of post-positivism and constructivism.“ The philosophy underlying this ‘bridge’ asserts that, if 
reality is assumed to be objective, it must still be interpreted by people (subjects) who are subjective 
beings. +35168.16° Constructivism describes how people sometimes called ‘research subjects’ work together 


with researchers to collaboratively construct meaning from data. 
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As our own direct experience of reality is inescapable, tools such as language (including that used in data 
analysis) offer ways to attempt to escape subjective experience and arrive at a collective 

experience. !”139) Through acknowledging the subjective nature of all enquiry, and attempting to justify 
methods and overcome or label bias where it might occur, the subjectivity of the case-study method used 


in this doctoral thesis is defensible.1°” 


In addition, in large organisations or other complex power structures (as described in the ASPREE study), 
critical realism offers a useful framework for analysing the interactions between people’s internal and 
external realities and the observer of those realities. Critical realism can be summarised as a way to 
explore causally relevant levels of reality, using multiple perspective to explore interactions case by case. 
171 A combination of both an interpretive and post-positivist approach therefore provided richer and 
more diverse data than using one of those methods alone. Using online research methods enabled me to 
adapt traditional discussion methods and co-create the methods described in this thesis to suit case study 
participants. Working with potential participants to co-design the study enabled a melding of objective 
and subjective data and using rigorous, transparent methods improved both the validity and richness of 
the data.**? Use of similar coding methods (including the STARDIT-PM categories) in the two online 


studies also improved the replicability and validity of the data.*’2 


Use of asynchronous text-based discussions in the online discussions also aligned with the participatory 
action research (PAR) paradigm, as it is more inclusive than other approaches. It also supports people’s 
different cognitive needs or preferences by providing access to information and discussion in a more 
accessible format.’”? This theoretical approach also enables everyone to be heard equally, in contrast to 
synchronous or face-to-face discussion that might be dominated by certain people. It is more inclusive 
because it is flexible and does not exclude people who, for example, might work full time or have caring 


responsibilities. 


The reflective practice case study of Chapter 8 ‘Involving Australian Indigenous peoples in precision 
medicine’ is distinct from the other three case studies, in that it is a post-event analysis that reports 
experiences, insights, debates and discussion from my individual perspective (checked by another 
researcher involved for validity).*6* The method and structure of reflective case studies is informed by the 


‘reflection cycle’, and is explained in more detail in the methods section of Chapter 8.7% 


Case selection 


Selection of cases for this thesis was informed by a number of factors, which were appraised using the 


following questions. ”* 
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1. Was it a population affected by genomics research, as distinct from the general public? 


2. Was it pragmatic, that is, was it possible to establish a mutually trusting and effective relationship 


within the time and resources available to the research project? 


3. Was the power dynamic equal and not exploitative (would the research offer participants something 


rather than just being passive subjects)? 
4. Were there conflicting or competing interests which could negatively affect the research? 


5. Was the proposed case study ethical, (including would the research be a burden on the populations’ 


or partner organisations’ capacities)? 


Cases studies can be defined as ‘typical’ or ‘deviant’, ‘influential’ or ‘extreme’, but these are one- 
dimensional labels that rely on the perspective of a single labeller and thus may change according to an 
individual’s perspective.*” To facilitate an understanding of cross-case characteristics, Table 3.4 explores 
and labels the characteristics of the case studies in this thesis from multiple perspectives. Using data 
available to the study team, each column explores a different feature of each case study. To make 
comparison with other cases more useful, the ‘population label’ groups the cases into a larger population 


grouping. For example, a population affected by a rare disease may be grouped with other rare diseases. 


The ‘representative features’ column explores features of the case that may be similar to others in the 
same population label, for example, with respect to equality of access to treatment. As deviance may 
affect data utility or generalisability, ‘deviant features’ explores how a case study may differ from others 
within the same population label. ‘Methods of involvement used’ outlines the participatory methods used 
to involve people during the case study. ‘Generalisable data’ explores what data from the case study may 
be able to be applied to other populations within the defined population label. Lastly, the ‘Usage’ column 


refers to how all data from the case study may be used. 


63 


Population Methods of involvement Generalisable 
Case study Representative features Deviant features Size Usage 
label used data 
a. ASPREE ‘Typical’ Large government funded Only healthy people over 70 years. | Representative on study 59 Learning from Confirm or 
cohort study | cohort study, with Study team using co-design team, telephone methods of disconfirm 
with biobanking and methods to explore acceptability interviews, face-to-face involvement and theories about 
associated international data sharing of future studies. events, co-authors on reported impacts. | methods of 
biobank. practices. paper. involvement. 
b. People affected | ‘Typical’ Charity started by person Charity had attempted to involve Representatives involved 26 Explore 
by Eosinophilic community affected, offering people in prioritising research in co-design, online likelihood of 
Gastrointest-inal | of shared information and support decisions prior to participatory surveys and online any causal 
Disorders (ausEE) | interest with an established online | action research. discussions, commented mechanisms 
defined by community. on paper. 
being 
affected by a 
rare disease. 
c. Shared ‘Typical’ Most people in the group Group is exceptionally large. A Representatives involved 18 
Ancestry group of made the discovery after member of the study team in co-design, online 
people who buying direct-to-consumer | discovered that their mother was surveys and online 
are donor genetic tests and making part of the group while discussions, commented 
conceived contact via online researching potential case studies. | on paper. 
half-siblings. | communities. 
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d. Indigenous ‘Typical’ The Aboriginal community Members of the Aboriginal Representatives involved N/A* 
Precision Medicine| group of is remote and experiences community have positive in co-design, face-to-face 
Initiative people from | multiple forms of inequity experiences of genetic medicine events, surveys. 

a remote and poor health outcomes. | and have previously been involved 

Aboriginal in another co-designed project. 

community. 


Table 3.4: Selected case study characteristics 


*the co-design process is ongoing and therefore the numbers of people involved are not quantifiable at this stage of the research 
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Unselected case studies 


Several case studies were considered for this doctoral research project. Appraisal of potential 
partnerships can be considered a proto-stage of co-design. Following such appraisal, it was decided 
not to partner with certain organisations in the co-design of research projects. So that candid and 
accurate explanation for these decisions can be provided, these organisations are not identified. The 
complexities of establishing a partnership relationship with an organisation, before discussion of 


project co-design can begin, are illustrated below. 


An Australian-based cystic fibrosis charity 

Exploratory meetings considered hosting online discussions for community members, particularly 
regarding prioritising drugs for commissioning. To inform the process, the study team also consulted 
an independent person affected by cystic fibrosis outside of the formal meetings with the charity. 
Internal reorganisation within the charity meant that the timing of the partnership would not be 
mutually suitable. In addition, pharmaceutical company funding of the organisation placed study 
team members with links to Cochrane in a potentially difficult position. Due to these problems in 


timing and ethics, this case study was rejected. 


An Australian-based comprehensive cancer centre 

Comprehensive cancer centres work to integrate genomics into care pathways for patients. 
Following an invitation to present at the Centre, a member of staff at the Centre approached me and 
suggested a potential partnership. After several meetings, the study team decided that it was too 
early in the project to involve people in the way discussed. However, as an independent consultant, | 
developed a training session for the Centre to help people (including cancer patients affected by rare 
cancers) understand how they could get involved with the work of the Centre. Project staff 
continued to share information about the project and relevant learning with the Centre. Staff 


changes at the organisation delayed further discussion about partnerships. 


At the end of this doctoral research project (September 2020), | was approached by the Victorian 
Comprehensive Cancer Centre and worked with them (in my capacity as Director of Science for All) 
on a project to support public involvement in precision oncology and scope how people affected by 
cancer and the wider public can be involved in personalised cancer care.1 While not a part of my 
doctoral research, learning from the doctoral research was applied to this work, and a STARDIT 
report in the form of a protocol was produced describing proposed plans for involving the public in 
the work of the cancer centre. While permission was given to share an outline of the work 
conducted on the Science for All website, owing to a confidentiality agreement, the majority of this 


work remains unpublished at the time of writing (November 2021). 
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An Australian-based cancer centre research cohort 

Meetings were held with staff working on a cancer research project with a focus on clinical 
application of research. The project had links with several other Australian-based genomics research 
organisations. The meetings explored ways of working together. Accountability for involving patients 
and other stakeholders in the research was emerging, but is as yet unclear. Staff changes during the 
project set up phase complicated communications and the case study was rejected on the grounds 
of timing. 

A rare cancer charity with organisations in Australia, UK and USA 

Face-to-face meetings held in Australia and the UK explored potential ways of working together. 
People affected by the rare cancer had formed three aligned patient organisations in Australia, USA 
and the UK and were funding their own researcher. The study team explored ways of working with 
the group, but concluded that the organisation was seeking a way to manage communications and 
decision making about, but not limited to, research within the organisation. Any discussion hosted 
by the study team would have had to encompass the full scope of the organisations’ concerns, which 
would have been out of scope of this research project. While it was a promising project, it was 
rejected owing to complexity and timing. To inform the design of other projects, the study team 
stayed in touch with staff from these organisations and shared information and learning about best- 


practice. 
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Selected case studies 
Table 3.5 provides an overview of the case studies selected for this research. 


Table 3.5: Overview of selected case studies 


Case study cohort Summary Method Data sources 
a. Healthy people The ASPirin in Reducing Events in Working in partnership with staff and participant 1. Diary 11. Interview 
over 70 (ASPREE) the Elderly (ASPREE) study is a advisors, we involved people in the planning of a 2. ASPREE Newsletter 12. Event planning feedback 
prominent longitudinal primary possible future multiple-generational cohort study, 3. Meeting records 13. Pre and post event 
prevention trial involving a large encompassing genomics and other types of research. | 4. Email discussions information and questions 
research cohort in Australia The work culminated in a face-to-face event, where! | 5. Reports on progress 14. Facilitation plan and 
(>16,000 elderly Australians). facilitated a session exploring people’s views and 6. Interview participant relevant reflections 
perspectives about involvement in genomic initial feedback 15. Event recording 
research. 7. Meeting about interviews | 16. Participant feedback about 
8. Interview data event 
9. Interview summary 17. Email summaries of event 
10. Meeting about event. 18. Notes from event 
19. Meeting notes. 
b. People affected by | The charity ausEE works to improve | Working in partnership with representatives from 1. Diary 
Eosinophilic the lives of people affected by rare | the organisation, an online discussion forum was 2. Emails and meeting notes 
Gastrointestinal eosinophilic disorders. created to explore views and perspectives about 3. Online pre-discussion survey 
Disorders (ausEE) genomic research of people affected by rare 4. Learning resources for participants and facilitators 
immunological eosinophilic gastrointestinal 5. Online discussion with participants 
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Case study cohort 


Summary 


Method 


Data sources 


disorders. Online discussions were co-facilitated and 
moderated by Jack Nunn and a representative from 
the organisation, Kylie Gwynne, who is also a co- 


investigator. 


Online discussion with facilitators 
Online post-discussion survey 


Follow up survey for facilitators. 


c. Donor-conceived 
siblings (Shared 


Ancestry) 


A group of related family members 
who share the same ancestor (a 


common sperm donor) were 


invited to participate in the project. 


As a group of (an estimated) >500 half-siblings 
around the world who share a common father, the 
group presents a unique opportunity for human 
genomic research. Potential participants were 
involved in designing the study and an expert 
facilitator (not a relative of any member of the 


group) led and moderated discussions. 


Diary 

Emails and meeting notes 

Online pre-discussion survey 

Learning resources for participants and facilitators 
Online discussion with participants 

Online discussion with facilitators 

Online post-discussion survey 


Follow up survey for facilitators. 


d. Indigenous 
Precision Medicine 


Initiative 


The Poche Centre for Indigenous 
Health is leading the ‘Indigenous 
Precision Medicine Initiative’. The 
Centre plans to work with 
communities to co-create research 
protocols for creating reference 
genomes to improve clinical 


translation of genomics research. 


This project will work in partnership by observing 
and assessing the process and standards set for the 
other case studies that form part of this doctoral 
research — including reporting planned activity as a 


STARDIT report. 


Meeting notes (including notes from face-to-face meetings 
with community members, with identifying information not 
recorded) 

Emails with study team 


Comments on shared documents. 
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Data summary and analysis 


The case study design used in this thesis was informed by international best practice, including use 


of standardised case study reporting to improve both quality and rigour.17°"”” 


In addition to quantitative analysis, qualitative data from each case study were analysed 
thematically in a number of stages including data mapping and familiarisation; transcription; manual 
coding; searching for themes; reviewing themes with team members; labelling and summarising 
themes; and reporting the findings.” An alpha version of the STARDIT tool was developed and used 
in parallel with the thematic analysis to organise data into pre-defined ‘super-categories’. Using the 
STARDIT reporting tool, this allowed consistent comparisons with other case studies and data to be 
made. Several other frameworks for reporting involvement in research also informed data collection 


and analysis.2°°”17218!_ Figure 3.2 summarises the stages of data analysis. 


Figure 3.2: Stages of data analysis 


Stage 1: Data mapping and familiarisation 


Stage 2: Coding 


Stage 3: Searching for themes 


Stage 4: Reviewing themes with study team members 


Stage 5: Labelling and summarising themes, data categorisation 
using STARDIT 


Stage 6: Summarising data and sharing analysis with 


participants; inviting feedback 


Stage 7: Reporting findings (publishing) 


Stage 8: Cross-case analysis 
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Involvement in genomics: Mapping what is known 


and unknown 


This section summarises the reviews which have been conducted as part of this doctoral research, as 
summarised in Table 3.6. In the following sections, | summarise a number of mixed-methods reviews 
that | conducted in an attempt to map both the knowledge and gaps in knowledge which exist 
around the questions of this thesis. Abstracts for the narrative review of public involvement and the 
scoping review of reporting frameworks and guidelines are included in the body of the thesis. The 
full reviews are presented in the Appendices. This allows for the scoping review of involvement in 
genomics research to be included in full as ‘Chapter 4 — Scoping review of involvement in human 
genomics research’. Table 3.1 above provides a summary of all the review methods, while Table 3.6 


below summarises the learning from each review. 


Who knows? 


Mapping both what is known and unknown is important for policymaking and science evaluation.*®* 


Angiology is the study of ignorance of that which we can never know (what is unknowable), while 
agnotology is the study of what we do not know, and why in the realm of what is knowable. +° The 
latter can be thought of as the social construction of ignorance. The best documented example is of 
people working for the tobacco industry who attempted to confuse a correlation between smoking 
and cancer by funding research into alternative causes, paying experts to promote smoking and 
producing pro-smoking ‘propaganda’ .**? Combined with systemic under-funding of research into 
causation, the World Health Organization projects that deaths from smoking will reach 10 million 
annually by 2030,1* partly caused by deliberate attempts to induce cultural ignorance about the 


risks. 


The study of culturally-induced ignorance is absolutely central to genomics research. For example, 
some observers have commented that “the world has placed excessive emphasis on the study of 
numerically and economically dominant European and Asian populations”, 1 with these populations 
over-represented in current databases of human genomes. This makes genomic medicine less useful 
for people from population groupings not represented in current databases, as any variations of 
known or unknown significance in these populations are not as well understood. The impact of this 


on Indigenous peoples is explored in Chapter 8, and in the paper | co-authored titled ‘Equitable 


expanded carrier screening needs Indigenous clinical and population genomic data’.° 
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The methods used for these reviews are informed by the ‘Routledge International Handbook of 
Ignorance Studies’.**° They attempt to map what is known about involvement in genomics research, 
what is not known, and why. Connected to these concepts is the ‘right to ignorance’, whereby some 
people choose not to know certain things about their own genome. Such preferences can influence 
the design of genomics research, including data sharing protocols. This is explored further in the 


case studies in Chapters 5 and 7. 
What do we know about people’s attitudes to genomics research? 


Extensive research about people’s attitudes towards participation in genomics research has been 
conducted. However, it remains the case that limited research has been done on people’s 


involvement in genomics research and any impacts of such involvement. 


A 2015 systematic review of stakeholders’ perspectives on biobank-based genomic research 
emphasised the importance to biobank participants of being informed about data-sharing practices, 
measures for privacy protection, and applicable sanctions if the data were misused.*?° Participants 
were generally very supportive of data sharing and considered that the benefits for science and 
society outweighed the potential risk of re-identification."”? This suggests that, unless there is 
evidence of serious data breaches, overly rigid data protection could constitute “paternalism” over 


patient preferences and be described as an “unjustified obstruction to research”.""° 


The UK’s 100,000 Genomes Project recognised that many people who joined as project participants 
did so because their participation would “benefit generations to come” .**”'"3) A 2014 study noted 
that this project can be viewed as potentially benefiting “anyone, anywhere”. Participants do not 
expect to benefit personally, and there is no sense that they are trading-off donation of data in 


exchange for direct benefits for themselves or their families. 186°® 


In a 2016 study, 80 per cent of people who had shared their genomic data said they did so in order 
to contribute to the advancement of medical research.®”"») In a 2020 global survey, most people 
viewed DNA data as different from other medical data. Notwithstanding, people were most willing 
to donate DNA data when the recipient was specified as a ‘medical doctor’. They were least willing 
when the recipient was a ‘for-profit researcher’.© Similar beliefs about participating in research in 
order to benefit society as a whole have been reported in a number of other countries, including the 


USA and Australia.18°-1%° 


While many surveys and reviews have focused on people’s perspectives about participating in 


genomics research, there is very limited information about their preferences with respect to how 
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they would like to be involved, that is, how they would prefer to share power in decision making, 
planning and analysis. For example, little is known about how people would like to influence 
decisions on data access or how they would prefer this to be achieved in practical terms. Such gaps 
in our knowledge have serious implications, in particular, for populations at greater risk of 
exploitation, such as Indigenous peoples whose culturally-specific preferences for involvement may 
not have been integrated into the planning of large genomics research projects. These populations 


are thus less likely to be involved in critical decisions about data sharing. 


73 


Table 3.6: Summary of learning from the reviews 


reporting and evaluating 


initiatives: A 


Review title Learning points 
How are the public 1. Language to describe involvement is varied and sometimes inconsistent. Standardised reporting would be advantageous. 
involved in health 2. Involving people in health research generally has beneficial outcomes for research and for health service planning. 
research, and what are the |3. Involving people as early as possible is considered best practice. 
impacts? A narrative 4. Anumber of high-impact actions can be taken that do not necessarily require considerable budgets or time. 
review (see Appendices) 5. Face-to-face involvement appears to involve greater cost in relation to impact, compared with other forms of involvement, including online 
methods. 
6. Involving people in planning how the public will be involved is crucial. 
7. Other than generic positive impacts, the most often reported impacts were improving the relevance of the research, improving dissemination of 
findings, and improving data collection. 
Public Involvement in 8. The methods to involve people reported by most initiatives were formal groups (63 per cent) and public events (41 per cent). 
Global Genomics 9. People performed diverse tasks when involved, including identifying research priorities, being a project co-investigator, participating in research 
Research: A Scoping design and management, sharing views about the research, and overseeing data access. 
Review?” 10. Initiatives reported various impacts from involving people, including a mobile outreach bus and improvements to governance frameworks. 
11. Specific genomics research tasks that people can get involved in included articulating phenotypes. 
12. Methods of involving people and tasks performed are varied. Evidence on effective ways of involving people is not clear, with data likely under- 
reported. Consistent reporting on and evaluation of involving people is required. 
Guidance for planning, 13. Guidance on reporting, planning and evaluating initiatives varies across disciplines. Many disciplines which share common methodologies use 


inconsistent language to describe similar methods, tasks and communication modes. 
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Review title 


Learning points 


multidisciplinary scoping 


review!? 


14. 


15. 


There is an urgent need to create standardised reporting of data about initiatives, including consistency of terminology to describe planning and 


reporting (including reporting impacts and outcomes), and consistent ways of sharing data to evaluate initiatives across disciplines. 


To facilitate machine learning, terminology needs to be translated consistently between human languages. The findings of this review support the 


continued co-creation of the reporting system ‘Standardised Data on Initiatives (STARDIT).*92 
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15 


16 


17 
18 
19 
20 


21 


22 
23 
24 
25 


26 


27 
28 


Narrative review: How are the public involved in health research 


more broadly, and what are the impacts? (Abstract) 


A more detailed summary of this narrative review is included in the Appendices. 


Before beginning research specifically into genomics research and involvement, it was important to 
explore existing literature and terminology on involvement in health research in general. The methods 
from this review informed the design of later reviews, as well as helping inform methodologies by 
exploring impacts from different methods, and identifying the need for something like STARDIT to be 
created. The main learnings from this review which informed this doctoral research were that language 
and terminology is inconsistent in the area of public involvement in research; systematic reviews called 
for improved reporting and consistency; online involvement methods had a lower cost and more 
impacts than face to face involvement; and participatory action research (PAR) was the method with the 
most reported outcomes. Thus participatory action research was chosen as the guiding paradigm of the 


doctoral research, and online methods of involvement were integrated into the research plan in 2017. 
Abstract 

Aim 

We wanted to understand how the public are involved in shaping health research and summarize the 
established methods of involving the public that might be applicable to genomics research. By looking at 


any impacts that this involvement might have had, we wanted to identify possible methods and 


approaches that may inform our future plans for involvement and impact assessment. 
Method 


A systematic search of systematic reviews relating to public involvement in research was conducted 
using boolean? operators in CINAL, Medline and Google Scholar. Data extracted from the reviews 
included what kind of involvement was taking place (the type), how it was done (the method), which 


stage of the research cycle it occurred and any impact on the research that might have occurred. 
Results 


This systematic narrative review summarises five systematic reviews identified in this area. Involvement 


was reported at each stage of research with mostly positive impacts reported at each stage. 


3 Boolean operators are used when searching to connect words together. The three basic boolean operators are: 
AND, OR, and NOT. They can help focus a search by connecting information 
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Involvement in data collection and analysis were the most reported stages of involvement. Agenda 
setting was the most frequently reported stage of the research cycle for involvement. Most of the 
methods of involvement described specified a method involving people in a group structure, rather than 
individually. A total of 27 different methods of involvement were extracted, with community based 
participatory research having the most impacts. Other than generic positive impacts, the most impacts 
reported were improving the relevance of the research, improving dissemination and improving data 
collection. The stages of research with the most reported impacts were ‘agenda setting’ and ‘data 


collection’, with dissemination the next highest. 
Conclusion 


Involving people in health research generally has advantageous outcomes for research and health 
service planning. Involving people as early as possible is considered best-practice. There are a number 
of high impact actions which can be taken which do not necessarily require considerable budgets or 
time. Face to face involvement appears to have an increased cost in relation to impact, compared to 
other forms of involvement, including involving people using online methods. Involving people in 
planning how the public will be involved is crucial. While there are limitations in the data available, it is 
clear that involving the people at any stage of research has value, with impact at all stages being 
possible if people are involved early in the research cycle. More work is needed to better document the 


impacts of involvement in research. 


Scoping reviews background 


A systematic scoping review was chosen as the method for exploring how the public are currently 


involved in global genomics research. 


The short section below is an extract from a peer-reviewed article | wrote with a co-author titled ‘What 
are Systematic Reviews?’ for the WikiJournal of Medicine*. The extract focuses on scoping reviews, and 


formatting and references have been incorporated into this document. The STARDIT report for this 


article is here: https://www.wikidata.org/wiki/Q101116128 
What are scoping reviews? 


Scoping reviews are distinct from systematic reviews in several important ways. A scoping review is an 
attempt to search for concepts by mapping the language and data which surrounds those concepts and 
adjusting the search method iteratively to synthesize evidence and assess the scope of an area of 


inquiry.?°21% This can mean that the concept search and method (including data extraction, 
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organisation and analysis) are refined throughout the process, sometimes requiring deviations from any 


protocol or original research plan.*%°1%° 


A scoping review may often be a preliminary stage before a systematic review, which 'scopes' out an 
area of inquiry and maps the language and key concepts to determine if a systematic review is possible 
or appropriate, or to lay the groundwork for a full systematic review. The goal can be to assess how 
much data or evidence is available regarding a certain area of interest. This process is further 


complicated if it is mapping concepts across multiple languages or cultures. 


As a scoping review should be systematically conducted and reported (with a transparent and 
repeatable method), some academic publishers categorize them as a kind of 'systematic review', which 
may cause confusion. Scoping reviews are helpful when it is not possible to carry out a systematic 
synthesis of research findings, for example, when there are no published clinical trials in the area of 
inquiry. Scoping reviews are helpful when determining if it is possible or appropriate to carry out a 
systematic review, and are a useful method when an area of inquiry is very broad”, for example, 


exploring how the public are involved in all stages systematic reviews.” 


There is still a lack of clarity when defining the exact method of a scoping review as it is both an iterative 
process and is still relatively new?®°. There have been several attempts to improve the standardisation 
of the method?9?19°.197.199 for example via a PRISMA guideline extension for scoping reviews (PRISMA- 
ScR)™. The International Prospective Register of Systematic Reviews (PROSPERO) does not permit the 
submission of protocols of scoping reviews”, although some journals will publish protocols for scoping 


reviews.1? 


This is the end of the extract from ‘What are Systematic Reviews?’ published in the WikiJournal of 


Medicine.*® 
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Guidance for planning, reporting and evaluating initiatives: A 


multidisciplinary scoping review 

In order to move towards standardised reporting of initiatives, | led this scoping review in order to show 
the current variation in guidance on planning, reporting and evaluating initiatives. The need for this 
review was identified by the co-design process for the STARDIT Alpha version, and informed by the 


learning from ‘Public involvement in global genomics research: A scoping review.” 


It is important to note that this review is not limited to one discipline and includes information from 
disciplines including health research, health technology assessment, environmental research, basic 
research, community based participatory research and educational research health, international 
development and community arts projects. The objective of this review is to display the considerable 
variation in this area in order to inform both future systematic reviews and proposed standardised ways 


of reporting data on initiatives.1°" 


This review is not intended to summarise, appraise or find consistencies in ways of planning, reporting 
or evaluating initiatives, nor is it an attempt to systematically map and compare all data. This review has 
been designed as a first step towards future systematic reviews which could explore the question more 
thoroughly. It is also designed to help appraise the need and feasibility of a standardised way of 


reporting initiatives across disciplines.1° 


The following extract is from the abstract for the article ‘Guidance for planning, reporting and evaluating 


initiatives: A multidisciplinary scoping review’, which is currently under review. 1% 


Abstract 


Background 


The principles of involvement in research and other interventions (and associated methodologies such 
as participatory action research and citizen science) are the same across all fields (health research, 
health technology assessment, environmental research, basic research, community based participatory 
research and educational research), information about initiatives and their impacts is not consistently 
reported across disciplines. Additionally, the linguistic variation between terms such as ‘involvement’, 
‘engagement’ and ‘participation’ makes it difficult to conduct systematic comparisons of how people 
have been involved in initiatives, and any impacts. Standardised Data on Initiatives has been identified 


as one solution to standardised data about initiatives. However, before attempting to create a multi- 
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disciplinary solution, it is important to map the current variation across disciplines for planning, 


reporting and evaluating initiatives. 
Methods 


The design of this scoping review was informed by the guidance on Preferred Reporting Items for 
Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) Checklist for this 
scoping review. Multiple information sources were included in the search. These included, EBSCOhost 
Research Databases, Academic Search Complete, Business Source Complete, CINAHL, Education 
Research Complete, ERIC, MEDLINE, MEDLINE (with Full Text). Any information source which included 
information about how an initiative was planned, reported or evaluated was initially included as 
‘guidance’, before being screened for relevance. Initiatives were included if they described an action, 


intervention, project or other kind of participatory process. 
Results and discussion 


An initial 20,868 results were refined and screened by two authors (SS, JN), with a sample checked by a 
third author. A total of 529 were selected for inclusion, with sources included from multiple disciplines. 
Guidance on planning, reporting and evaluating initiatives exists across disciplines, with most being 
health research and some being environmental research. For initiatives which span disciplines, it is 
often not clear which methods of planning, reporting and evaluation are most appropriate, for example, 
an initiative to reduce air pollution to improve public health would span the disciplines of public health, 
environmental studies, policy and potentially education. The variation in language and associated 
concepts makes cross discipline comparison of methods extremely difficult and almost impossible to do 


in a systematic way. 
Conclusions 


Guidance on reporting, planning and evaluating initiatives is varied across disciplines. Many disciplines 
which share common methodologies use inconsistent language to describe similar methods, tasks and 
communication modes. There is an urgent need to create standardised reporting of data about 
initiatives, including consistent terminology to describe planning, consistent terminology for reporting 
(including reporting impacts and outcomes) and consistent ways of sharing data in order to evaluate 
initiatives across disciplines. Additionally, terminology needs to be consistently translated between 
human languages to facilitate machine learning. The findings of this review support the continued co- 


creation of the proposed reporting system ‘Standardised Data on Initiatives (STARDIT).1%2 
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This is the end of the extract from the article ‘Guidance for planning, reporting and evaluating initiatives: 


A multidisciplinary scoping review’, which is currently under review.*** 


Mapping the gaps in knowledge 


How the reviews informed this doctoral research 


The results from reviews conducted for this doctoral thesis provided an important context for this 
research. This short section summarises the main learning points from the reviews regarding current 
gaps in knowledge and practice. Learning from the narrative review informed development of the 
scoping review and, subsequently, the initiation of STARDIT. Development of STARDIT was informed by 
methods from the scoping review, with other interested people and organisations invited to be part of 
the co-design process. The Alpha version of STARDIT, created to elicit feedback on the proposed 
framework, identified the need for guidance on planning, reporting on and evaluating initiatives, 
including genomics research initiatives. The literature review informed the development of the STARDIT 
Beta version. The interdisciplinary nature of public health genomics showed that STARDIT needed to 
work outside of the specific ‘medical’ context, and the co-design process highlighted the need for an 


interdisciplinary solution to reporting involvement in research and other initiatives. 
How do we know how people are involved? 


There is currently no standardised way to describe, report on and evaluate the impacts of involvement, 


27,167,201-205 According to a 2018 systematic review, existing 


making it difficult to establish best practices. 
tools for reporting public involvement in health research such as GRIPP2 do not appear to have been 
widely adopted,” especially in low and middle-income countries.””” Furthermore, tools like GRIPP2 are 
only appropriate for reporting on English language health research and offer no structured or publicly 


searchable databases for analysing report data.” 


No existing tools for reporting involvement include indicators of involvement that are consistently 
described or reported, and the terminology used varies around the world. Subjectivity is evident in 
definitions of indicators of involvement. To consistently assess and report on involvement, objective 
measures need to be developed. Some concepts related to involvement cannot be reduced to linguistic 
labels such as ‘ethics’ and other values, so objective measures (or indicators) of who was involved and 


how in co-defining such concepts is important. 
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What evidence is there about the most effective ways of involving 
people? 


Lack of consistent reporting about involvement in genomics research mean that it is not currently 
understood which methods, techniques and approaches are optimal, most acceptable, and preferred by 
the public, researchers and other stakeholders.’ Evidence is therefore lacking on the best ways to 
integrate involving people as a routine part of the research cycle. In addition, many researchers are 
unfamiliar with the notion of involving people in their research, or are reluctant or ill-equipped to begin. 


Standard tools and guidance on effective ways of involving people are rarely available to them. 
What are people’s preferences for involvement? 


While there are multiple ways to plan and report on people’s participation in designing involvement in 
initiatives, °”2°%??2 there is no standardised way to plan, report and evaluate this data. Plans for 
involving people must meet the needs of researchers (be realistic within time, budget and other 
resources) and the people they are trying to involve. Involving people at the outset in designing how the 


public and other stakeholders will be involved in an initiative can help ensure these needs are met. 
Why Standardised Data on Initiatives (STARDIT) was created 


The creation of Standardised Data on Initiatives (STARDIT) and the STARDIT Preference Mapping tool 
(STARDIT-PM) were developed in response to the gaps identified above. The next section summarises 
how STARDIT was co-designed, co-created and resulted in a Beta version, in which it is now possible to 
report data about initiatives (including genomics research) in a standardised way across multiple human 
languages. This includes being able to describe how people were involved, in which tasks, and any 


impacts of the involvement. 
Standardised Data on Initiatives (STARDIT) 


Learning from the reviews described in this section inspired me to initiate and design ‘Standardised 


Data on Initiatives’, and begin a co-design process to refine it. 
Context for STARDIT co-creation 


The need for the creation of a system such as STARDIT was identified during both the narrative review 
and the scoping review of public involvement in genomics research. Results from both reviews showed 
that reporting of involvement was inconsistent, with varied terminology even within Anglophone 


countries, as well as between them. 
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The narrative review (and other literature) identified a participatory method as the most appropriate for 


the development of a system such as STARDIT. 


In addition, while designing the research projects for this thesis it became apparent that the relatively 
small numbers of people would be participating in two of the case studies (the Shared Ancestry study 
and the ausEE study). This meant that extrapolations from the data would be improved if data reporting 
was standardised across the multiple case studies in order to consistently combine the results with 


other studies for comparison’™”, facilitating a cross-case analysis from standardised data. 
STARDIT co-creation 


After | had the initial idea for STARDIT, | decided to proceed a co-design process. | consulted experts in 
the field, including the then Chief Operating Officer of INVOLVE (then part of the UK’s National Institute 
for Health Research), a Chief Editor of the journal Research Involvement and Engagement and the Chief 
Editor of the Wiki/ournal of Science. One suggestion included renaming it from ‘Standardised Reporting 
on Dissemination, Involvement and Translation’ to ‘Standardised Data on Initiatives’. | designed the co- 
design process, and used this process to co-create the Alpha version, with more detail provided in the 
report ‘STARDIT Public Consultation Report — September To December 2019’, which is included in the 
Appendices’. Detailed information about the subsequent co-creation process is included in the 


STARDIT Beta version. A visual summary of the co-design process can be seen in Figure 3.3. 


Figure 3.3: Visual Summary of STARDIT co-design process 
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How STARDIT was used in this doctoral research 


STARDIT was used throughout this doctoral research, with learning from using the Alpha version 
(version 0.1) used to inform the development of the Beta version (version 0.2). STARDIT 0.1 was used to 
map preferences for involvement, plan involvement and case study, report case study method and data, 
compare data from each case study and evaluate impacts and outcomes. In addition a public domain 
STARDIT Beta version 0.2 report with structured machine readable linked-data is stored in the public 


domain and included in each published case study. 
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Standardised Data on Initiatives — STARDIT: Beta Version 


The remainder of this chapter has been submitted for peer-review and is currently under review as of 
16" December 2021. A pre-review version of this article is available online, identical to the one below. 


The article and the additional files for this article can be found at the link can be found at this link:® 


https://doi.org/10.21203/rs.3.rs-934285/v1 


Figure and table numbers in the remainder of this chapter have been changed to align with this thesis. 
Please note, as the additional files for the STARDIT Beta version are over 25,000 words long, they have 


not been included in this thesis to reduce the file size. They can be accessed via the link above. 
Authors 


Jack Nunn, Thomas Shafee, Steven Chang, Richard Stephens, Jim Elliot, Sandy Oliver, Denny John, 
Maureen Smith, Neil Orr, Jennifer Preston, Josephine Borthwick, Thijs van Vlijmen, James Ansell, 
Francois Houyez, Maria Sharmila Alina de Sousa, Roan D Plotz, Jessica L Oliver, Yaela Golumbic, Rona 
Macniven, Samuel Wines, Ann Borda, Hakon da Silva Hyldmo, Pen-Yuan Hsing, Lena Denis, Carolyn 


Thompson 


Please note, affiliations can be viewed at the above link. 
Plain English summary 


All major problems, including complex global problems such as air pollution and pandemics, require 
reliable data sharing between disciplines in order to respond effectively. Such problems require 
evidence-informed collaborative methods, multidisciplinary research and interventions in which the 
people who are affected are involved in every stage. However, there is currently no standardised way to 
share information about initiatives and problem-solving across and between fields such as health, 


environment, basic science, manufacturing, education, media and international development. 


A multi-disciplinary international team of over 100 citizens, experts and data-users has been involved in 
co-creating STARDIT to help everyone in the world share, find and understand information about 


collective human actions, which are referred to as ‘initiatives’. 


STARDIT is an open access data-sharing system to standardise the way that information about initiatives 


is reported, including information about which tasks were done by different people. Reports can be 
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268 updated at all stages, from planning to evaluation, and can report impacts in many languages, using 
269 Wikidata. STARDIT is free to use, and data can be submitted by anyone. Report authors can be verified 
270 to improve trust and transparency, and data checked for quality. 

271 

272 STARDIT can help create high-quality standardised information on initiatives trying to solve complex 
273 multidisciplinary global problems. Among its main benefits, STARDIT offers those carrying out research 
274 and interventions access to standardised information which enables well-founded comparisons of the 
275 effectiveness of different methods. This article outlines progress to date; information about submitting 


276 reports; planned next steps and how anyone can become involved. 
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Abstract 


Background 


There is currently no standardised way to share information across disciplines about initiatives, 
including fields such as health, environment, basic science, manufacturing, media and international 
development. All problems, including complex global problems such as air pollution and pandemics 
require reliable data sharing between disciplines in order to respond effectively. Current reporting 
methods also lack information about the ways in which different people are involved in initiatives, 
making it difficult to collate and appraise data about the most effective ways to involve different 


people. 


STARDIT (Standardised Data on Initiatives) was created to help everyone in the world find and 
understand information about collective human actions, which are referred to as ‘initiatives’. STARDIT 
enables multiple categories of data to be reported in a standardised way across disciplines, facilitating 
appraisal of initiatives and aiding synthesis of evidence for the most effective ways for people to be 


involved in initiatives. 


Method 


STARDIT is the first open access web-based data-sharing system which standardises the way that 
information about initiatives is reported across diverse fields and disciplines, including information 
about which tasks were done by which stakeholders. STARDIT data is licensed as public domain (CCO) 
and integrated into Wikidata; it works across multiple languages and is both human and machine 
readable. Reports can be updated throughout the lifetime of an initiative, from planning to evaluation, 
allowing anyone to be involved in reporting impacts and outcomes. 

STARDIT development is guided by participatory action research paradigms, and has been co-created 
with people from multiple disciplines around the world in multiple ways, working with Indigenous 
people, cancer patients, health researchers, environmental researchers, economists, librarians and 


academic publishers. 
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Results 

Over 100 people from multiple disciplines have been involved in co-designing STARDIT. STARDIT reports 
have been created for peer-reviewed research in multiple journals and other research projects. A 
working Beta version was publicly released in February 2021 (ScienceforAll.World/STARDIT). 
Conclusions 

This article outlines progress to date on STARDIT; information about submitting reports; planned next 


steps and how anyone can become involved. STARDIT can help create high-quality standardised 


information on initiatives trying to solve complex multidisciplinary global problems. 
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Introduction 


Background 


Many problems facing life on earth transcend the capacity of any single discipline to address. For 
example, problems such as pandemics, air pollution and biodiversity destruction cannot be 
characterised solely as ‘public health’, ‘environment’ or ‘education’ problems.2?*2"4 Solving such 


215 


problems calls for holistic approaches** and will require governments, industry, research organisations 


and people around the world to work in partnership. 


People need access to valid and reliable information to make informed decisions”, which typically 
requires evidence. Depending on the context, this evidence-informed approach is called ‘research’, 
‘evaluation’2”’, ‘international development’, ‘education’ or an ‘initiative’. Hereafter all of the above will 
be referred to as ‘initiatives’. For example, when deciding a response to a pandemic, standardised data 
can improve retrieval of relevant information which can be used to inform which affected individuals or 
organisations could be involved in the design of the response and which outcomes are most important. 


This can include deciding which stakeholders should be involved in which tasks, such as prioritising 


outcomes. 


In this article we explain how Standardised Data on Initiatives (STARDIT) builds on work to date by 
standardising a wide variety of data in a format applicable across multiple sectors, disciplines and 
languages. It is hoped that the creation of this evidence base will add to understanding and evaluating 
what works, for whom, why, and in what circumstances.2”**7®?"8 Hereafter, data generated by an 
initiative (including raw data), information about the data (meta-data) and information about the 


initiative will all be referred to as ‘data’ unless otherwise specified. 


In 2020, the United Nations Secretary-General stated that ‘purposes that involve data and analytics 
permeate virtually all aspects of our work in development, peace and security, humanitarian, and human 
rights’, encouraging ‘everyone, everywhere’ to ‘nurture data as a strategic asset for insight, impact and 
integrity — to better deliver on our mandates for people and planet’”’. Similarly, the United Nation’s Paris 
Agreement highlighted the critical role of ‘sharing information, good practices, experiences and lessons’ 
in response to preventing irreversible climate change.?? While organisations such as Cochrane (health) 
and The Campbell Collaboration (social sciences) are working to create high-quality systematic reviews 
of medical, social and economic initiatives, there remain limitations to the data available for such 


reviews. After a recommendation from the Organisation for Economic Co-operation and Development 
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(OECD), successful data sharing initiatives in biodiversity exist, such as the Global Biodiversity 
Information Facility (GBIF)?2°, however there also remain limitations and accessibility issues in sharing 


and standardising biodiversity data.?22222 


It is often essential to include those affected by initiatives in the design and delivery of those 
initiatives.” For example, with an initiative to respond to a pandemic, those creating and delivering an 
initiative, and those affected by the outcome may be the same people. Forms of participatory action 
research where anyone can be involved in any aspect of research (including amorphous terms such as 
‘citizen science’) are increasingly recognised as crucial paradigms for solving such global problems such, 
as they can help ensure that initiatives are aligned with the priorities of those affected.” However, 


while the importance of involving people is clear,*4 evidence-informed methods of doing so are 


lacking.27167.201-204 


A recent statement defined a role for the public in ‘data intensive’ health research.?? While in the 
health research disciplines there are over 60 different tools for reporting or supporting public 
involvement, most published tools are not used beyond the groups that developed them, and none 
work across multiple disciplines.” Current reporting methods also lack information about the ways in 
which different people are involved in initiatives, making it difficult to collate and appraise data about 
the most effective ways to involve different people. In addition, ‘citizen science’ and ‘participatory 
action research’ are blurring the lines between concepts such as ‘researcher’, ‘public’, ‘patient’ and 
‘citizen’. 27,130,226-229 

In addition to providing new standardised data categories for describing who was involved in which 
tasks of an initiative, STARDIT can also incorporate the many existing data standards (see the 
supplementary resources ‘Using Standardised Data on Initiatives (STARDIT): Beta Version Manual’), thus 
creating a unifying system for data hosting, linking and analysis. STARDIT can also report any different 
‘interests’ of stakeholders and the ways power is shared between different stakeholders. The word 
‘stakeholders’ here includes the public, those who have important knowledge, expertise or views that 


should be taken into account and others with a ‘stake’ in an initiative.2°*° 


Stakeholders can also include people who have financial, professional, social or personal ‘interests’. An 
‘interest’ can include a kind of commitment, goal, obligation, duty or sense of connection which relates 
to a particular social role, practice, profession, experience or medical diagnosis and may. 7°°. These can 
include financial or other interests which may compete or conflict with ‘public interest’. For example, 


a systematic review found that industry funded research is more likely to have outcomes favouring 
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those with financial interests who are sponsoring the research.!3®23t Other examples include people 
from certain sub-populations (including those from populations more likely to be exploited??), 
Indigenous peoples, or people affected by rare diseases may have a personal interest in initiatives 
relevant to those specific populations, separate to the ‘general public’.+”?57 For example a person with 
a rare disease may have a personal ‘interest’ in research into a treatment for that disease.” STARDIT 


allows standardised reporting of stakeholders and any interests. 


Sharing data in a consistent way may help ensure that benefits of initiatives are shared more equitably 
(for example, by improving accountability).”” In addition sharing information about who ‘owns’ or 
controls access to data and how such data access decisions are made can help people make informed 
decisions about participating in research.’ By reporting involvement in initiatives, STARDIT also allows 
acknowledgement of people otherwise excluded from the public record — such as patients, people 
donating personal data, medical writers, laboratory assistants, citizen scientists collecting or analysing 
data, custodians of traditional or Indigenous knowledge, translators, interviewers, coders and code 


reviewers. 
Objective 


The objective of STARDIT is to address current limitations and inconsistencies in sharing data about 
initiatives. The STARDIT system features standardised data reporting about initiatives, including who has 
been involved, what tasks they did, and any impacts observed. STARDIT is designed to support a culture 
of partnership across disciplines and is, wherever possible, aligned and interoperable with existing 
reporting models and frameworks such as those used in health, environment, manufacturing , 
publishing, government policy, education, arts and international development (see Table 3.7). In 
addition, the STARDIT Preference Mapping (STARDIT-PM) tool provides a standardised way to report 
information about different stakeholders’ preferences, including preferences for power-sharing and 
methods of involving people during an initiative. STARDIT data is licensed as public domain (CCO) and 
integrated into Wikidata, which is collaboratively edited structured data.*?? The working Beta Version of 
STARDIT uses Wikidata to enable definitions to be co-created by contributors anywhere in the world, 
and therefore works across human languages, with interoperability with other platforms planned for 
future versions. A link to the working Beta version can be found at: 


ScienceforAll.World/STARDIT/Beta2?? 
Current usage 


STARDIT provides a way to report data about who did which tasks in an initiative. STARDIT reports have 
also been used to describe a number of research projects, including data about who did which tasks, 


91 


425 
426 
427 
428 
429 
430 
431 
432 
433 
434 
435 
436 
437 
438 
439 
440 
441 
442 


4443 


444 
445 
446 
447 
448 
449 
450 
451 
452 
453 
454 
455 
456 
457 
458 


ethics approval, funding, methods and outcomes.*?3+235 In health research, projects which have used 
STARDIT reports include participatory action research projects involving a large cohort study of >15,000 


236 a protocol for precision medicine for Aboriginal Australians?, 


healthy, elderly research participants 
and a group of patients and families affected by a rare immunological disorder’, and a project involving 
extended family of donor-siblings who share the same sperm-donor father.*”2” An environmental 
research project has also used STARDIT to report the initiative, which works with citizen scientists to 
locate critically endangered species using eDNA.7?*3° Australian Genomics have created a prospective 
STARDIT report to describe planned work to improve guidance on involving the public in genomic 
research.” In medicine, STARDIT is already being used by the Wikipedia-integrated open access peer 
reviewed WikiJournals, which has articles which are integrated into Wikidpedia.?° For example, a 
STARDIT report has been created to share information about a Wiki Journal of Medicine article about 
systematic reviews (with an associated integrated Wikipedia page)*®, including information about 


authors, editors and peer-reviewers.7 This allows readers to critically appraise the source before 


deciding whether to use or share it. 


Finally, STARDIT is being used by the Cochrane Consumer Executive (CCE) to report the process of co- 
creating a ‘Values Statement’ for Cochrane (internationally)?**, as an organisation and community of 


volunteers. The CCE will report the planned process, the actual process and any impacts using STARDIT. 
Potential applications 


STARDIT’s potential applications are summarised in Table 3.7. Among the principal applications, 
STARDIT offers public access to standardised information which enables the comparison of methods 
with the most impacts, such as ways of involving stakeholders in initiatives. The United Nations defines 
assessing impact as ‘establishing cause and effect chains to show if an intervention has worked and, if 
so, how’.?® With more data being shared, STARDIT could support decision making when planning 
stakeholder involvement in initiatives, and enable more people to assess the rigour of impact 


assessments.2? 


For example, many industries use self-regulatory processes to govern industry practices, with examples 
including the Forest Stewardship Council (FSC), Marine Stewardship Council (MSC)?*4, Certified B 
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Corporations,*” and multiple Good Manufacturing Practice (GMP) guidelines. STARDIT could be used to 
improve public awareness of, and access to the data already reported by such self-regulatory standards. 
Increased transparency could, for example, support people to make informed decisions when investing 

or buying products; automate analysis of data to facilitate such decisions, and improve accountability 


overall. 
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In addition, STARDIT could be used to share information which makes research more reproducible?*®4’, 
improving accessibility to the information required to critically appraise research and evidence and thus 
improving trust in processes such as the scientific method***9, and facilitate an appraisal of different 
knowledge systems, including Indigenous knowledge systems.”°° Such data sharing could also improve 
the translation of trusted, quality research and data, by empowering people to both access and appraise 
relevant data. For example, improved access to more standardised information (in multiple languages) 
about data and outcomes, could help to facilitate more informed collaborations between researchers 
and those monitoring and protecting critically endangered species, particularly where there is no 


common language.”>) 7°? 


Defining ‘initiative’ and ‘involvement’ 


As STARDIT is designed to report data across disciplines, distinctions between concepts such as 
‘intervention’, ‘research’, ‘project’, ‘policy’, ‘initiative’ (and similar terms) are of secondary importance 
compared with communicating ‘the aims or purposes of specified actions’; ‘who did which tasks or 
actions’; ‘are there competing or conflicting interests’, and the ‘outcomes from a specific action’. In this 
way, STARDIT can be used to report on any kind of collective action, which can include interventions, 
projects or initiatives — including a clinical study, education interventions or any kind of 
evaluation.??725425> In this article, we use the word ‘initiative’ to describe any intervention, research or 
planned project which is a kind of collective human action. We define ‘involving’ people as the process 
of carrying out research, initiatives or interventions with people, rather than on them.?® Involvement 
occurs when power is shared by researchers, research participants, and other relevant stakeholders 
(such as the public, industry representatives and experts). While meanings of these terms are often 
imprecise and can be used interchangeably, ‘involvement’ here is distinct from ‘engagement’, which is 
where which information and knowledge about initiatives is shared, for example, with study 


participants who remain passive recipients of interventions.” 7°? 


Using and developing data standards 


The current Beta Version of STARDIT maps terms and concepts using the Wikidata initiative (part of the 
Wikimedia Foundation)*°, which includes definitions (taxonomy), a way of describing relationships 
between concepts (ontology)?’”, and a system to translate definitions and ontology between many 
languages. Examples of existing taxonomies include the National Library of Medicine’s Medical Subject 


Headings (MeSH), which are used extensively in multiple kinds of literature reviews." 
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How to involve people in combining or merging overlapping taxonomies for different subsets of data 
has been identified as an important question in the process of taxonomy.”©*! By using Wikidata, 
STARDIT can be used by anyone to store both publicly accessible data and meta data (data about data), 
and link to hosted structured linked data. While STARDIT is a novel element set, where possible it will 
also incorporate element sets from established data standards and map them where possible (see Table 
5 in Additional file for examples of data standards which could be incorporated). This includes standard 
elements and value sets and controlled vocabularies.”* The terms used in this paper are working terms, 


which will be progressively standardised over the lifetime of the project. 


Structured Wikidata can help define terms and concepts clearly and unambiguously, in a transparent 
and open way. For example, colours in the spectrum are described by a standard numerical code in 
Wikidata, whereas the names of colours change according to different languages. Also, people with 
different DNA variations will also experience some colours differently. Similarly, the Wikidata entry for 
‘patient’ has the human-readable definition of ‘person who takes a medical treatment or is subject of a 
case study’ (translated into 54 other languages) and a machine-readable definition consisting of dozens 
of semantic links to and from other Wikidata entries.” The terms ‘participant’ and ‘research participant’ 
are similarly coded, defined and translated. For terms that do not currently exist in Wikidata (for 
example, ‘biobank participant’), a definition can be contributed by anyone in any language, refined by 
other users, then coded and translated into multiple languages by Wikidata. Developing taxonomies and 
ontologies will be an ongoing process facilitated by the current Wikidata infrastructure, and may require 


creating additional tools to create more inclusive ways of involving people in developing taxonomies.”° 
Methods and paradigms 


Participatory action research 


STARDIT development is guided by participatory action research (PAR) paradigms, which guides 
initiatives by aiming to involve all stakeholders in every aspect of the development and evaluation of an 
initiative.?76263 Participatory research is a form of collective, self-reflective enquiry undertaken by 
people in order to understand their situation from different perspectives.’?” Development has also been 
influenced by existing work in health research, including the multidisciplinary area of public health, 
which incorporates social, environmental and economic research. In a health context, participatory 
research attempts to reduce health inequalities by supporting people to be involved in addressing 
health issues that are important to them, data collection, reflection and ultimately in action to improve 


their own health.?28 At the core of participatory research is ‘critical reflexivity’. The process asks people 
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involved to reflect on the causes of problems, possible solutions, take any actions required which might 


improve the current situation, and evaluate the actions.?2° 


Rights-based paradigm 


The United Nations (UN) Universal Declaration Human Rights states everyone should be able to ‘receive 
and impart information and ideas’.’t The UN also states that democracy, development and respect for 
all human rights and fundamental freedoms are interdependent and mutually reinforcing’.°* To uphold 
human rights and ‘environmental rights’2°, and for ‘the maintenance of peace’, people require ‘media 
freedom’ in order to ‘seek, receive and impart information’”’, free of unaccountable censorship. 
STARDIT has been created in order to help anyone uphold these universal rights, by providing a way to 


share open access information in a structured way with a transparent process for quality checking. 
Cultural neutrality 


Values, assumptions, ways of thinking and knowing are not shared universally. The participatory process 
used for developing STARDIT required and will continue to require that it attempts to map cultural 
variations, in an attempt to avoid unconsciously reinforcing particular (often ‘dominant’)™ values. 
Transparent acknowledgement of differing values and perspectives is critically important, in particular 
when mapping if different stakeholders’ values are complimentary or opposing. A participatory process 
requires mapping all of these perspectives and, where possible, involving people in labelling different 
perspectives and values. For example, STARDIT has already been used to map the varying perspectives 


of multiple stakeholders when planning a multi-generational cohort study.® 


Many problems facing humans are shared by non-human life forms and ecosystems, including rapid 
climate change, air pollution and sea-level rise. If initiatives are to operate in inclusive, culturally-neutral 
ways, reconsideration of the language used to describe relationships between humans, non-human life 
and the environment is essential.2°° Environmental and social sciences are challenging and redefining 
colonial-era concepts of what can be ‘owned’ as property or who ‘owns’. 7°26 As a result, ecosystems 
such as rivers and non-human animals, are being assigned ‘personhood’.”°” 2° For example, a public 
consultation by a ‘dominant’ group might ask, ‘who owns the rights to the water in a river system?’ 
This question imposes the dominant group’s values on people who may not share the same concept of 
‘ownership’. In this way, Western European legal and economic traditions are frequently incompatible 


with those of some Indigenous peoples’ 265270271 


The participatory process used for developing STARDIT has attempted to be transparent about how 


different stakeholders have been involved in shaping it in order to improve how the system can be used 
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to map values and provide more culturally neutral guidance for planning and evaluating involvement in 
initiatives. However, it is acknowledged that it will be a challenging process to ‘de-colonialise’ and ‘de- 
anthropocise’ language and action?’*273, as this may be perceived as a challenge to some people’s 
cultural attitudes which may not align with the United Nation’s universally enshrined principles of 
democracy, human rights and environmental rights. In addition, ongoing co-design will be required to 


ensure STARDIT is as accessible and inclusive as possible. 
Development phases and methods 


Both the STARDIT Alpha version (0.1) and the Beta version (0.2) have already involved many people 
from diverse disciplines and backgrounds in the development, as this is integral to its effectiveness 
(Figure 3.3). It has been co-created using methodologies informed by PARTICIPATORY ACTION 
RESEARCH and other health research reporting guidelines.” Participatory action research describes 
related approaches which involve experts (such as researchers), the public and other stakeholders 


“working together, sharing power and responsibility from the start to the end of the project”.°*° 


The Alpha version of STARDIT (version 0.1) followed the recommendations of a 2019 scoping review led 
by Nunn et al, which mapped public involvement in global genomics research.’ This review stated that 
‘without a standardized framework to report and transparently evaluate ways people are involved, it 
will be difficult to create an evidence base to inform best-practice’.’” This review was followed by an 
additional review conducted in 2020 led Nunn et al, which mapped international guidance for planning, 
reporting and evaluating initiatives across multiple disciplines.*°* STARDIT was also informed by a 


4,234,235 


number of participatory action research projects, and a report for the Wikimedia Foundation by 


the charity Science for All.?”° 


The charity Science for All has hosted the co-creation process since 2019. Science for Allis a charity 
based in Australia which support everyone to get involved in shaping the future of human knowledge, 
with co-created values guiding their work.2”° Development was informed by a number of literature 
reviews and guidelines, with methods of involving people in the development of STARDIT guided by the 
Enhancing the Quality and Transparency of Health Research (EQUATOR) network’s approach to 
developing reporting guidelines.2”*?”” Methods of involving people included public events, online 


discussions and a consultation process. 


The co-creation process is being supported pro-bono by Science for All, and has also received in-kind 
support from the EPPI-Centre and individuals from multiple organisations worldwide. A modified Delphi 


technique was used at some stages, with this method to be reviewed when co-creating future 
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versions.?’®?79 Many people were invited to provide feedback on all aspects of STARDIT, including its 
feasibility, design and implementation. They could comment anonymously using online forms and 
shared documents, in online discussion forums, via email or during face-to-face or video meetings. 
Science for All hosts an online working group which continues to guide the development of STARDIT 
according to the terms of reference.”®° Anyone is welcome to join the working group, contribute to 
discussions and vote on decisions and ensure alignment with other initiatives. More detailed 
information about the consultation process for the Alpha and Beta versions up to May 2021 can be 
found in a 2020 and 2021 report. 18t Further information about who was involved in the Beta Version 
development and proposed future development phases can be found in the supplementary 
information. STARDIT and all associated work and logos are currently published under the Creative 
Commons Attribution-ShareAlike 4.0 International license (CC BY-SA 4.0)°?, with the quality of any 
future iterations being the responsibility of not-for-profit host organisations and future licensing 


decisions to be made transparent, with anyone invited to be involved. 


Figure 3.3: Visual Summary of STARDIT co-design process 


2019-—— eer ee. 2020 > 2021 


Stage 2: Public Consultation Stage 3: Beta version develop 


August September — September Oct-Dec 2020 2021 


o iy | i 
STARDIT co- Draft version Alpha Combinedfaceto Global feedback Beta version Feedback collated 
designed witha shared for version face and video on nen eed developed and and incorporated - 
small team of comment shared as discussions hosted sions ae rE content shared as in STARDIT Beta 
experts pre-print in in London public report public domain submitted for peer 
public review 


Co-design led by Science for All with global stakeholders involved in the following tasks at each stage: 


S a PN £8 CN H STARDIT Beta 


Detailed feedback Feedback on every Feedback on every aspect of Beta version, 
on feasibility and aspect of Alpha including technical implementation 
implementation version 


Version One implementation 


Once STARDIT Beta (version 0.2) has been submitted for publication, work will begin on the next version 
(version 1.0). Those involved with STARDIT development will disseminate information, gather feedback 
and recruit more people and organisations to participate as project partners and potentially funders. 
This stage is estimated to take between 2 to 3 years, at which point a working group will formally invite 
other appropriate partner organisations (such as the UN and WHO) to adopt the STARDIT framework. A 
Steering Group will be established to oversee and continually improve the STARDIT system. STARDIT will 


require continued working with publishers, research funders and governments to encourage adoption 
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of the reporting tool. More detail on the proposed next stages can be found in the section 


‘Development phases’ of the Additional File 1, of the version of this article currently under review (as of 


1st November 2021).® 


Scope and applications 


STARDIT is the first and only data-sharing system that enables standardised sharing of data about how 


people are involved in any type of initiative, across any discipline, including involvement in the planning, 


evaluation and reporting of initiatives. In addition it allows comparison of both evaluation methods and 


any impacts or outcomes in relation to standardised terminology. Table 3.7 summarises the proposed 


scope and potential applications of the STARDIT. 


Table 3.7: Example applications of STARDIT 


Health research 
Social research 
Genomics research 


Environmental research 


Health and social policy 


Other government policy 
(transport, arts, education, 


environment etc) 
Foreign policy 


Proposed policy (including 
draft policy and 


manifestoes) 


International development 


Reporting: Funding, conflicting or competing interests, co- 
design, experts involved, people affected involved, 
methods, process for deciding and measuring outcomes, 
protocols, who is accountable for ensuring protocol is 
followed, information about data storage, sharing, 
ownership and custodianship, information about data 
security practices and standards, information about 
consent and withdrawal processes evaluation of entire 
research process, ethical review, information about data 


analysis and data validation 


Reporting: Values of people involved, sources of data and 
evidence, data on past and current initiatives and 


283 process for policy (or proposed policy) 


spending 
creation, process for deciding and measuring outcomes, 
experts involved, people affected involved, policy or 
manifesto writers, conflicting or competing interests of 
people involved, purpose of policy (what needs have been 
identified, how and by who), outcomes from policy 


(including outcomes measured by those affected by 


j policy), policy evaluation (reporting if it achieved what was 
intended) 


Education and Educational initiatives Reporting: Sources of data and evidence for intervention, 


learning purpose of intervention, process for educational 
intervention creation, funding, conflicting or competing 
interests, experts involved, people affected involved, 
conflicting or competing interests, process for deciding and 
measuring outcomes, outcomes from intervention, 


evaluation of intervention, ethical review 


Arts Community arts projects Reporting: Purpose of project, process for project design 
and implementation, experts involved, people from 
communities intended to benefit involved, funding, 
conflicting or competing interests, process for evaluating 


project, project evaluation, project outcomes. 


Arts funding Reporting: People involved in deciding funding process, 
purpose of funding, people allocating funding (funding 
sources), funding amount, conflicting or competing 
interests, process for deciding outcomes of funding, 


evaluating the funding allocation process 


Reporting: People involved in researching, writing 
(including medical writers), creating, reviewing (including 
peer reviewers), disseminating and funding, information 
about any potential risks (to human health or lifeforms, 
natural or cultural heritage), information about who 
assessed those risks and how (for example, medical 
information standards”), information about consent to 
appear in images and verified appearances of public 
figures, information about ownership of data or knowledge 
(including concepts of intellectual property, copyright and 


license information, relevant blockchains and non-fungible 
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Intangible cultural heritage 
(including folklore, 
traditions, language), 
traditional, local and 
Indigenous knowledge and 


wisdom 


Tangible cultural heritage 
(including cultural 


property???) 


Hardware designs 
(including hardware 


architecture, device 


tokens), evaluating knowledge translation, reporting 


impacts and outcomes.””° 


Reporting: who created any content containing the 
Indigenous or traditional knowledge, what tasks they had, 
how this knowledge was shared and any relevant concepts 
of ‘owning’ or ‘property’; reporting who knows certain 
things (for example, people who are recognised as 
‘Preservers of Important Intangible Cultural Properties’ 2®°); 
reporting who is recognised as an Elder, community leader, 
Indigenous elders or leaders (and by who); reporting who 
does or does not have permission to verify, share, redact 
or edit content (including stories, beliefs, cultural practices 
and medicine)2*; information about data custodianship?”°, 
information about any potential risks (to human health or 
lifeforms, natural or cultural heritage); information about 
who assessed those risks and how, information about 
informed consent process, information about any cultural 
sensitivities or restrictions (including relevant information 
about gender, clan, tribe or other culturally constructed 
groupings)*°” 39, information about relevant laws and 
lore*°°, ethics processes (including who was involved and 
how), reporting impacts and outcomes from 


dissemination.2”° 


Reporting: Who was involved in creating the property, any 
concepts of ownership or guardianship in relation to the 
property, data about ongoing management (including 
monitoring, exhibiting, restoring or moving), data about 
cultural significance and stakeholders involved in defining 


this 


Reporting: Who was involved in creating the designs and 
how, who reviewed them and how (including relevant 


safety, regulation or standards information), what formats 


are the designs shared as and in what medium, 


information on license(s), outcomes and impact of the 


hardware 


Reporting: who created code (including algorithms), who is 
involved in reviewing and scrutinising code (including who 
is involved in which ethical review processes), what code is 
part of which distinct projects or forks, what language the 
code is in, what medium (for example, machine or DNA), 
information about ownership of data or knowledge 
(including concepts of intellectual property and copyright), 
information on license(s), purpose of code, outcomes and 


impact of the code 


Reporting: data about who was involved in service design, 
monitoring and management processes, data about 
funding for monitoring or management (for example, 
funding for pollution monitoring), data about how 
information will be stored and shared (including what will 
be redacted and data security), data about who decides 
what data will be redacted and how this decision is made, 
information about how data will be analysed (including 
relevant code and algorithms) and how learning from data 
will be shared, information about relevant data privacy 
legislation and regulation 


Process evaluation Reporting: data about processes (industrial, public health, 


organisational)**’”, people involved, outcomes 


Evaluation of participatory Reporting: data about participatory research methods and 


methods (co-design) compare outcomes. 


Transparent rating Reporting: Processes of transparency rating (or ‘scoring’) 
data quality about initiatives based on how much 


information about the initiatives is shared in a publicly 
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Production, 


consumerism 


and business 
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Industry standards 


‘Green’ industries and eco- 


tourism 


Infrastructure, 


construction and interiors 


Finance and financial 


services 


Donation and philanthropy 


accessible way (or reasons for redaction, including 


Indigenous knowledge). 


Reporting: Internal processes and data sharing practices of 
self-regulating industry standards (for example, the Forest 
Stewardship Council, Marine Stewardship Council?“ and 
Certified B Corporations*“’), data sharing principles, 


process evaluation (including by those affected). 


Reporting: Transparent process for defining ‘green’ and 
‘eco’, experts involved, people affected involved, process 
for deciding and measuring outcomes, outcome measures, 


evaluation of process. 


Reporting: Transparent reporting of sources of building 
and furniture materials, such as wood (including relevant 
DNA information to verify sources of timber), metals and 
other materials (including information verifying the supply 
chain is slavery free), data from building and structural 


assessments 


Reporting: who is involved in decision making (including 
investment and divestment), who scrutinises decision 
making, who is involved in holding individuals to account 
and who scrutinises this process, competing or conflicting 
interests of people involved in decision making, data about 
how concepts such as ‘ethical investments’ are defined, 
impacts or outcomes from investments or donations, data 
sharing practices and security practices, data about who 


scrutinises security practices. 


Reporting: Any stated purposes or caveats for donation, 
organisations or individuals donating, how money was 
spent, who was involved in deciding how it was spent, 
what was the method for deciding this, who is accountable 


for overseeing this, any outcomes or impacts. 


Other products Reporting: Experts involved in production, other people 
medial device: involved in production process, resources involved in 


: production process (including relevant DNA information to 
electronics) 


verify products from plants, animals and fungi), 


e e Food ingredients, funding for resources (for example 


human use or 


Medicines demonstrating it is ‘slavery free’), process reporting 
ingestion (including Good Manufacturing Practice), regulation and 
Products for Food authorisation processes (for example medicines and 
non-human — medical devices), code and algorithm checking (for 

Medicines 
lifeforms example, autonomous vehicles) process for designing 

Other products impact assessment, impact assessment (including human 

and environmental), experts involved in dismantling 
process (including recycling), other people involved in 
dismantling process and disposal, evaluation of product 
according to transparently decided outcome measures 
Health Assessment process for Reporting: Process for deciding health technology 
Technology pharmaceuticals, devices, assessment (oversight and scrutiny), sources of data and 
assessment procedures and evidence, process for deciding and measuring outcomes, 
organisational systems experts involved, people affected involved, conflicting or 
used in health care competing interests, outcomes from assessment decisions 
(including outcomes measured by those affected by 
assessment decisions), collation of adverse event reports 
from Governments and reputable sources, assessment 
evaluation (did it achieve what was intended?), results of 
economic evaluations 
Health and Health care and services Reporting: Process for assessing needs (including who was 
social care and involved, the method and budget), process for 
services prioritisation of services (including budgets and ‘rationing’ 


Social care and services 
decisions), process for designing and implementing service 
or care (including who was involved, the method and the 
Other services : ; i 
budget), process for evaluating service or care (including 


impacts), patterns for evaluating service improvement 
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initiatives, process for reporting adverse events and 
malpractice (including the overview and scrutiny of this 
process), process for identifying patterns of sub-optimal 
service, process for responding to malpractice or other 
identified issues, process for identifying impact indicators 


(including geolocation data) 
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Further examples of how STARDIT can be used are provided in the supplementary information, 
including; using STARDIT in genomic research for mapping phenotypes and reporting who was involved 
in helping define and describe them; providing data to critically appraise information sources (including 
public videos); report data about case studies consistently; creating ‘living systematic reviews’ and 


training machine learning from STARDIT data. 
Using STARDIT 


Across all disciplines, ‘plan’, ‘do’ and ‘evaluate’ are recognised as distinct stages of initiatives?**. While 
there are many ways to involve different people in these stages, standardised reporting and thus 
evidence-informed methods of doing so are lacking.*®””4 Figure 3.4 describes how STARDIT can be used 
to map how people might be involved in designing, doing, reporting and evaluating initiatives, starting 


with ‘idea sharing’. 


Figure 3.4: Planning and evaluating initiatives using STARDIT 


Idea sharing © Report planned initiative 


pedate 7 
Í a Fa O Preference Mapping 


© a 
Update Musee) Designing and Report updated plan 
funding 


Foote | 
Do initiative 
Me 
Data collection a 
Update and analysis O Update 
Reporting initiative design in STARDIT 


Questions such as, ‘Who decides how people are involved?’ and, ‘Who is involving whom?’ and ‘what 
are people’s preferences for ways of working’ can be difficult to answer and is an active area of 
research”??? For example, planning a healthcare initiative requires input from experts as well as from 


the people the initiative is intended to help*®. Figure 3.5 summarises a way of using STARDIT to report 
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646 the design process of initiatives, with Table 3.8 providing details about how involvement from different 
647 stakeholders can be reported at different stages. The section ‘Detailed reporting of design using 
648 STARDIT’ in the supplementary resource ‘STARDIT Manual Beta Version’ provides more comprehensive 


649 information. 


650 Figure 3.5: Reporting initiative design in STARDIT 
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652 Table 3.8: Summary of reporting initiative design in STARDIT 


Initiative stage Data reported 


Stage 1: Idea identified: An idea for an intervention, project or research is A Report planned initiative 


identified and articulated 
Stage 2: Idea refined 


The idea is refined with a small group of stakeholders?®87130,134,293-296 


Stage 3: Stakeholder mapping: Existing stakeholders attempt to map who O Breference Mapping 
is included and who might currently be excluded from the process}30181 

Stage 4: Co-create communication plan 

Develop a communication plan to invite people to co-create involvement?*°293:297 


Stage 5: Share plan: Share the idea (according to the communication O) Report updated plan 
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and ask for feedback on it (including the involvement plan) 184152298 

Stage 6: Analyse feedback: Collect and analyse feedback, share results. 297°) 

Stage 7: Finalise idea and involvement plan: Co-create the plan (including O ii plan 
the plan for involving people), seek relevant permissions and ethics 79930 
Do initiative (see ‘Planning and reporting initiatives using STARDIT’) 


Stage 8: Evaluate involvement: Evaluate the process and the impact of SS. 
O Report end of initiative 


both the initiative and involving people in the initiative 
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Mapping preferences for involvement 


Involving multiple stakeholders in designing how people should be involved in initiatives is considered 
best practice, as it may facilitate power sharing and improve the process overall.?”30t Current 
explanations of participatory research methods, and the language used to describe them, vary 
considerably. There is no agreed, consistent way to describe how people have been involved ina 
research initiative, or to report the impacts of their involvement. 

The STARDIT Preference Mapping (STARDIT-PM) tool provides a standardised way to report the 
preference of multiple stakeholders. Anyone can be involved in creating a STARDIT report, which means 
that data on the impacts and outcomes of participation can be contributed by diverse stakeholders. 
Such reports will help researchers make informed decisions when planning participation in research. 
For example, a recent study showed how a charity for people affected by a rare disease involved a small 
number of people affected by the rare disease in discussing preferences for how best to involve the 
wider community of people affected in future research prioritisation and planning’. Those involved had 
a good understanding of any specific needs or preferences for involvement, and shared preferences for 
the tasks (such as overseeing data access), method (facilitated discussions) and mode of involvement 
(online text-based discussion). The STARDIT-PM data about this processes showed a preference for 
being involved using online discussions, and the STARDIT report stated that involving people influenced 
the way the charity planned to involve people prioritising research in the future.234 

Examples of completed STARDIT-PM can be found in the additional files of a number of research 
projects*?3436, Table 3.9 summarises questions which can be asked to map stakeholder preferences 


with respect to involvement in initiatives. 


The first stage of preference mapping requires individuals to self-identify as belonging to a specific 
grouping of people. People from that grouping then share views on how people from other groupings 
could be involved (or which groupings should not be involved). For example, labels for such groupings 
could include: 

e only people with a professional role in the initiative 

e everyone (any member of the public who is interested) 

e anyone who might be indirectly affected by the initiative 

e only people who are directly affected by the initiative 

e only people who are participating in the initiative 


e only people with a financial interest in the initiative. 


As a consistent mapping system for use across all initiatives, STARDIT would allow both comparison of 
diverse stakeholder views and exploration of similarities and variations in relation to preferences for 


involvement. Used alongside other planning tools, this information could help align initiatives with 
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stakeholders’ preferences. In this way, how stakeholders are involved throughout an initiative could be 
co-designed from the outset. Analysis of the data about preferences should involve stakeholders from 
multiple groupings to ensure that a diversity of perspectives are involved in assigning meaning to any 


data. 
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Table 3.9: Questions for mapping preferences for involvement 


Question 


Which stakeholder group does this person 


align with? 


Describe any financial relationship or other 


interest this person has to this project 


Views on the purpose and values of the 


research 


Describe how you think the learning from 


this initiative could be used 


Views on who data from this project should 


be shared with and how 


Views on who should be involved (which 
‘groups’ of people) — including who should 
not be involved — following answers may be 


categorised depending on the stakeholder 


group 


Views on specific tasks of this person or 


group 


Preferred modes of communication 


Views on what methods should be used 
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Rationale for question 


To establish which grouping(s) the person identifies as 
being part of — for example ‘researcher’ or ‘participant’ 


(noting any groupings should be co-defined) 


To provide a public record of any potential conflicting or 


competing financial interest 


To establish the purpose of the research, and the 
motivations and values of the initiative from multiple 


perspectives 


To establish views about knowledge translation and 


application of learning 


To establish that person’s view about data sharing and 


ownership 


To establish that person’s views on which ‘groups’ of 
people they think should be ‘involved’ in research — that 
is, having a role in shaping the research design, direction 
and outcomes Note: Answers may require sub- 
categories if there are multiple categories for who 


should be involved 


To establish that person’s views on the tasks of the 


specific stakeholders who they think should be involved. 


To establish that person’s preferences on 


communication modes with stakeholder groups 


To establish that person’s views on which methods 
should be used to involve people — for example ‘online 


survey’ 


696 


Question 


Views on facilitators of involvement 


Views on barriers of involvement 


Views on what the outcome or output of the 


involvement could be 


Views on which stage of the research this 


group should be involved? 
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Rationale for question 


To explore that person’s perceptions of what might 
facilitate involving specified groups of people and help 


inform the design of involvement 


To explore that person’s perceptions of what might be a 
barrier to involving specified groups of people and help 


inform the design of involvement 


To ascertain the expectations of that person about what 


involving the specified groups of people might achieve 


To establish that person’s views on which stage of the 
research the specified groups of people should be 


involved in 


Beta Version interface 


The data fields in the STARDIT system are summarised in Tables 3.10, which presents the full version 
of the data fields. The ‘Minimum Contribution Reporting Form’ (MICRO) specifies the minimum 
information required to make a STARDIT report and these fields are highlighted in the table and 


marked with an asterisk (*). 
Authorship 


Acknowledging those involved in reporting ensures accountability for accuracy and increases trust in 
report content. STARDIT reports must be completed by named people who are accountable for the 
data being reported. Ideally, a public persistent digital identifier (for example, an ORCID number)? 
or an institutional email address will be linked to authors’ names using Wikidata. 

Reports cannot be completed anonymously, but STARDIT editors can redact author details from 
publicly accessible reports for ethical reasons (such as privacy or risks to safety). 

Report authorship can be led by any stakeholder, including people associated with, or affected by, 
the initiative such as employees, researchers, participants, or members of the public. The affiliations 


of people formally associated with the initiative can be shared in a report. 
Submission and Editorial process 


Reports can currently be submitted to STARDIT via a simple online form or emailed as a document 
file. Editors review content for quality control (for example, checking that publicly accessible URLs 
align with the data in the report), but will not critically appraise the initiatives or methods. The 
Editorial process is currently parallel to the WikiJournal process, involving selected Editors from 
these journals. While Editors will not approve the ethics of the initiative, a transparent process for 
considering ethical issues will be considered before publishing a report. The Editors may consider 
questions such as, ‘Does data need to be redacted in order to prevent harm and protect or preserve 
life?’ or, ‘Is personal information being shared without consent?’ For more information about the 
Editorial process for reviewing data quality and ethical considerations, see the section ‘Editorial and 
peer review of STARDIT reports’ of the supplementary resource ‘STARDIT Manual Beta Version’. 
Once approved by the Editors, the STARDIT data will be entered into the database in a machine- 
readable format using structured data, based on the widely used Resource Description Framework 
(RDF) developed by the World Wide Web Consortium (W3C), which is used by Wikidata.2°? Each 
STARDIT report is assigned a unique Wikidata item number and all previous versions are navigable in 


a transparent history. 
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In future versions, it is proposed that stakeholders will be able to submit reports directly via an 
application programming interface (API) which will facilitate machine automation of STARDIT report 
creation). In addition, machine learning algorithms could be programmed to generate STARDIT 
reports from existing databases. As humans and machines submit reports, categories or meta-tags 
will be suggested (such as ‘patient’, ‘member of the public’), with the option of adding, or co- 
defining, new categories using the Wikidata system for structured data.°“ 
The database will generate a unique version number for the report with a Digital Object Identifier 
(DOI). To create an immutable version, the report will also be using the Internet Archive (a charity 
which allows archives of the World Wide Web to be created, searched and accessed for free)*”. 
Finally, the report will be assigned a status, with the data quality checking being described as: 

e manually added, no human review (low quality checking — no DOI assigned) 

e machine added, no human review (low quality checking — no DOI assigned) 

e human review (medium quality checking — DOI assigned pending Editorial decision) 

e peer or expert reviewed, with publicly accessible sources for indicators and references 


checked (higher quality checking — DOI assigned pending peer or expert review). 


Processes for data checking and assigning report status need to be further developed and agreed by 
the STARDIT working group. For example, developing a transparent process if a report has been 
created about an initiative with no involvement from anyone associated with the project, or only 
one subset of stakeholders. In such cases, the Editorial team might give a short period of time for 


any other stakeholders to be involved in checking and editing any information. 
Updating reports 


STARDIT will enable reports to be updated as initiatives progress over time. Updates will be 

reviewed by the STARDIT Editors. Once an update is approved, the system generates a new version 
number, while also preserving the original report. Updates might include, for example, information 
about involvement in the initiative, or about dissemination, translation, co-creation of new metrics 


to assess impacts, or longer-term outcomes.?!? 
Table 3.10: Summary of STARDIT Beta Version data fields 


A minimum dataset is required for a STARDIT report. This is called the Minimum Contribution Report 
(MICRO) and the required categories are highlighted and marked with an asterisk (*). Relevant 
Wikidata items and qualifiers for these fields are provided in Additional File 1, in the section 


‘Developing taxonomies and ontologies’, and on the Science for All STARDIT Beta webpage.??3 
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Core: Initiative 
context - This 
information locates 
the initiative within a 


clear context. 


Identifying information 


Relevant publicly accessible URLs/URIs 


Other identifiers (e.g RAiD*™, clinical trial 


(pe) 


Keywords or metatags — including relevant 


search headings (e.g. MeSH?®?) 


Other relevant information (free text) 


Status of initiative 


Methods and paradigms 
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Date range (start and end dates of initiative) 


Include any information about theoretical or 
conceptual models or relevant ‘values’ of people 
involved with this initiative, including any 


rationale for why certain methods were chosen 


Report authorship — 
Information about 
who completed the 


report and how 


Please note this 
section can be 
completed multiple 
times if there are 


multiple authors 


Identifying information for each 
author (authors can be 


anonymised in the public report 


but at least one verified identity 


will need to be sent to STARDIT 
Editors to attempt to prevent 


falsified reports) 


Tasks in report completion 


Other information 


Accountability 


Date 


Date of report submission (automatically 


generated) 


Input: Ethics 


assessment 


Ethics approval information (if 


applicable) 


Approval date and approval ID - include any 


relevant URL 


Input: Human 
involvement in 


initiative 


Details about how each group 
or individual was involved in 


the initiative 
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Section 


Data category 


Data field 


Who is involved in 
this initiative and 


how? 


Editors assessing 
involvement may 
need to use the 
STARDIT ‘Indicators 


of involvement’ tool 


participants’. Additional ‘labels’ or ‘meta-tags’ 


to describe people may be added if appropriate. 


How many people were in each grouping label? 


Tasks of this person or group (list as many as 
possible)* — including any information about 
why certain people were included or excluded in 


certain tasks (such as data analysis) 


Method of doing task? How did these people 
complete these tasks? (what methods were 
used) — for example ‘group discussion’ or 


‘reviewing documents’ 


Communication modes? What modes of 
communication were used — for example, ‘group 
video calls’, ‘telephone interviews’ or ‘postal 


survey’ 


How were people recruited, contacted or 


informed about these tasks? 


Involvement appraisal 


Methods of appraising and analysing 
involvement (assessing rigour, deciding 


outcome measures, data collection and analysis) 


Enablers of involvement (what do you expect 
will help these people get involved — or what 


helped them get involved) 


Examples of enablers 


Barriers of involvement (what do you expect will 
inhibit these people from getting involved — or 


what inhibited them from getting involved). Are 
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there any known equity issues which may 


contribute? 


Examples of barriers, and any attempts to 


overcome them 


How did the initiative change as a result of 
involving people? For example, did the initiative 


design or evaluation plan change? 


Note: this can be answered separately for 


different individuals or groupings of people 


Involvement outcomes, 


impacts or outputs 


Learning points from involving | What worked well, what could have been 
people improved? Was anything learned from the 


process of involving these people? 


Stage Which stage of the initiative were these people 
involved? (please provide information about any 
distinct stages of this initiative, noting some 


may overlap) 


Financial or other interests 
(including personal or 


professional interests) 


Financial What was the estimated financial cost for the 


initiative. 
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Section 


Data category 


Data field 


Input: Material 
involvement in 
initiative 

Mapping financial or 


other ‘interests’ 


Funding information (link to publicly accessible 
URL if possible) - this may include the project 
funder, funding agreements, grants, donations, 
public ledgers, transaction data or relevant 


block(s) in a blockchain 


Time How much time was spent on this project 
Note: this can be answered separately for 
different individuals or groupings of people 

Other Describe any costs or resources that cannot be 


measured financially or quantitatively - this may 
include expertise, traditional or Indigenous 


knowledge, volunteer time or donated resources 


Outputs: Data 


including code, 
hardware designs or 
other relevant 


information 


Sensitive data 


Secure criteria 


Data adheres to relevant industry/discipline 


data security requirements 


Repository How is data entered, changed or removed 
within a repository? 

Usage Who is the data from this initiative shared with? 
Who has access to sensitive data and how is this 
decided? 

Safety Is data encrypted? Is it anonymised or de- 
identified? What methods are used for re- 
identification? What is the risk of unauthorised 
re-identification? 

Open data FAIR criteria Data adheres to FAIR criteria” 
Findable Describe relevant metadata, how the data is 


machine readable and other relevant 


information 
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Section 


Data category 


Data field 


Accessible 


How can data be accessed — include any 
information about authentication and 


authorisation 


Interoperable 


How is data interoperable or integrated with 
other data? Include information about 
applications or workflows for analysis, storage, 
and processing, and resulting file formats or 


other outputs 


Reusable How can data be replicated and/or combined? 
Indigenous CARE Data adheres to CARE principles?”?*° 
data principles 
Collective How will Indigenous Peoples derive benefit from 
Benefit the data 
Authority to How will Indigenous Peoples and their 
Control governing bodies determine how relevant data 
are represented and identified 
Responsibility | How will those using the data provide evidence 
of these efforts and the benefits accruing to 
Indigenous Peoples 
Ethics How have Indigenous Peoples’ rights and 
wellbeing been centred during the data life 
cycle 
All data Hosting Where is it data stored and hosted -share any 
location data if appropriate 
Owner Who ‘owns’ the data or claims any kind of 
copyright, patent(s), or other specific types of 
intellectual property - include relevant open 
licensing information 
Analysis Describe methods used to analyse the data 
methods (including a link to any relevant code and 


information about validity) 
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Section 


Data category 


Data field 


Usage 


How can data be used? /nclude information 


about license and attribution 


Dissemination 


How is information about this data 
disseminated? For example, how are results 


from analysis shared? 


Impact 


impact/effect of the output 


Data control 


Who controls access to the data? How are 
decisions about data access made? Is data 
anonymised or de-identified? What methods 
are used for re-identification? What is the risk 
of unauthorised re-identification? How is this 


risk managed? 


Management 


and quality 


Which person (or organisation) is responsible 


for managing (or ’curating’) the data? 


Who is accountable for ensuring the quality and 
integrity of the data? (this may be an individual 


or organisation) 


Impacts and 
outputs: 
Publications, events, 
changes, learning 


items etc. 


What was learned 


What new knowledge has been generated? (if 
appropriate, include effect size, relevant 


statistics and level or evidence)* 


Knowledge translation 


Describe how the learning or knowledge 


generated from this initiative has or will be used 


Impacts 


Have there been any outcomes, or has anything 
changed or happened as a result of this initiative 


that isn’t captured in previous answers?* 


Measurement and evaluation 


How has or how will this be measured or 


evaluated? 


Who is involved in measuring or evaluating this? 
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Section 


Data category 


Data field 


Who was or is involved in deciding on the 
outcomes used to evaluate any impacts or 


outcomes? How were they involved? 


Information completed by Editors 


STARDIT report version number (assigned) 


Report number assigned to distinguish it from 


any future updated reports 


Indicators 


completed by Editors 
and/or peer 


reviewers 


Editors and peer 
reviewers assessing 
the report will need 
to look for indicators 
in the following 
categories on 
publicly accessible 


URLs* 


Indicators of involvement 


Use the STARDIT ‘Indicators of involvement’ tool 


Indicators of data practice 


compliance 


Use the relevant criteria 


Indicators of translation and 


impact 


Other indicators 


Discussion and future versions 


STARDIT is the first system that enables sharing of standardised data about initiatives across 


disciplines. By using Wikidata, STARDIT will make use of existing infrastructure capable of co-defining 


types of data in multiple languages7***"*>"*. The co-design process (hosted by the charity Science for 


All) ensured people from multiple organisations and countries were involved in both creating and 


refining STARDIT, ensuring that it is usable and relevant in multiple disciplines. Consultation with 


experts, and source materials from around the world, have informed the design of STARDIT. Co- 


authors come from disciplines including health research and services, environmental research and 


management, economics, and publishing, with over 20 different institutions represented. Future 


versions should be informed by a regular, systematic search, review and appraisal processes, using 
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the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) data set?*?, used 


for reporting in systematic reviews and meta-analyses. 


STARDIT enables reporting of who was involved, any impacts of stakeholders’ involvement, and 
outcomes of initiatives over time. This functionality addresses a serious limitation of the current 
peer-reviewed publication process in which articles are not easily updated. However, there is no 
single process for making decisions that would improve and refine the processes, language and 
taxonomies associated with reporting initiatives, including who was involved in which tasks.?2* 
Similarly, based on feedback from Indigenous community leaders, patient representatives and 
others, it is essential to ensure access to learning and development opportunities is available to 
support people to both access and create STARDIT reports. The STARDIT project therefore needs to 
continually appraise the inclusiveness and effectiveness of its multidisciplinary, multilingual system, 


including accessibility of interfaces. To achieve this, the project will continue to work with its partner 


organisations, including the Wikimedia Foundation, a global leader in this field. 


While there are multiple methods for mapping values,?*>"* there is currently no agreed, 
standardised way to map the values (beliefs and personal ethics) of those involved in initiatives and 
those creating reports in STARDIT. Further research is needed to facilitate mapping of values and 


detect whether certain perspectives are being consciously or unconsciously excluded. 


STARDIT seeks to be an easy-to-use way for people from multiple disciplines to share data about 
initiatives. However, amassing sufficient reports to create a useful database is estimated to take at 
least 5 years, and will likely require machine learning. For example, adversarial machine learning 
may be used in parallel with humans (for verifying data) to generate STARDIT reports from existing 
publicly accessible data at a scale and speed otherwise impossible for humans alone to achieve’. 
Similarly, the process of creating ‘living systematic reviews’ from STARDIT reports is currently 


theoretical and would require significant development and rigorous testing to realise. 


It is important to note that access to Wikidata is actively blocked by governments or internet service 
providers in some countries. While such censorship limits people’s ability to contribute or critically 
appraise data, STARDIT has been designed to be both interoperable with existing standards, and 
‘future proofed’ to allow interoperability with existing and emerging data systems beyond Wikidata. 
Science for All will continue to host the co-creation process and to monitor and evaluate the project. 


However, an open, transparent governance process that enables anyone to be involved in decision 
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making and ongoing co-design of STARDIT will need to be established, and is proposed in the 


supplementary resources. 


Ensuring that the STARDIT development process is inclusive and ethical, and that the database is 
quality assured, is paramount to ensuring that STARDIT is credible, useful and trustworthy. STARDIT 
currently relies on volunteers and pro-bono services from not-for-profit organisations. In the future, 
people should be paid for certain tasks, especially if the project is to avoid excluding the involvement 
of those from lower socio-economic backgrounds who may not be able to afford to volunteer their 
time. For the success and longevity of this project, a sustainable, transparently decided funding 
model needs to be established, which ensures both the independence of the data, the hosting 


process and the governance. 
Conclusion 


This article summarises work to date on developing Standardised Data on Initiatives (STARDIT), an 
open access web-based data-sharing system for standardising the way that information about 
initiatives is reported across diverse fields and disciplines. It outlines ways it has currently been used 
by different initiatives, and the next stages of development in detail. In accordance with the 
principles of transparent participatory action research, the authors invite the involvement of any 
interested persons in developing and improving the next version of STARDIT, Version 1.0. Detailed 


and up-to-date information about STARDIT is available on the Science for All website. **® 


This is the end of this chapter, and the content which has been submitted for peer-review at this link: 


8 


https://doi.org/10.21203/rs.3.rs-934285/v1 
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Chapter 4 — Public involvement in 
global genomics research: A scoping 
review 


This chapter is adapted from the published, peer-reviewed 2019 review titled ‘Public involvement in 
global genomics research: A scoping review’.?” Please note: 
e Figure numbers have been changed to align with the format of this thesis, and some 
formatting has been adapted, including integrating references into this thesis. 
e This paper uses the term ‘facilitators’, where later chapters use the term ‘enablers’ to avoid 
confusion with those facilitating discussions. 
e The published version of this article can be found here, along with the supplementary 


material: https://doi.org/10.3389/fpubh.2019.00079 
Authors 
Jack Nunn, Jane Tiller, Peter Fransquet, Paul Lacaze. 


Abstract 


Public involvement in research occurs when the public, patients or research participants are actively 
contributing to the research process. Public involvement has been acknowledged as a key priority for 
many prominent human genomics research initiatives worldwide. However, to date, there has been 
no detailed analysis or review of the features, methods and impacts of public involvement occurring 
in human genomics research projects worldwide. Here, we review the reported public involvement 
in 96 human genomics projects (initiatives), based on a database of initiatives hosted by the Global 
Alliance for Genomics and Health, according to information reported on public domain websites. To 
conduct the scoping review, we applied a structured categorization of criteria to all information 
extracted from the search. We found that only a third of all initiatives reported public involvement in 
any capacity (32/96, 33%). In those reporting public involvement, we found considerable variation in 
both the methods and tasks of involvement. Some noteworthy initiatives reported diverse and 
comprehensive ways of involving the public, occurring through different stages of the research 
project cycle. Three notable initiatives reported a total of eight distinct impacts as a result of 


involving people. Our findings suggest there would be intrinsic value in having more public 
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involvement occur in human genomics research worldwide. We also suggest more systematic ways 


of reporting and evaluating involvement would be highly beneficial, to help develop best practices. 
Introduction 


In human genomics, there is a growing need to increase the involvement of the public in research 
and policy development, as this has been identified as a crucial aspect of responsible research 
practice. 7°** The concept of ‘public involvement’ in research is defined as research that is carried 
out ‘with’ people rather than ‘on’ them.?°6 Public involvement can also be defined as when the 
public, patients or research participants actively contribute to the research or policy development 
process.329 

The number of people involved in genomics research is predicted to grow substantially in coming 
years.*>*6 By 2025, it is estimated that nearly 2 billion people worldwide will have had their DNA 
sequenced, creating a global imperative for responsible and effective public involvement in 
research.?° Many high-profile genomics research initiatives have already made public statements 


about the importance of involving people, with some governments positioning public involvement as 


a democratic right.223>*”. 


The benefits of involving the public in research are wide-ranging. They include improving trust and 
public influence over research?®?+35; ensuring that research is conducted in an ethical, accessible and 
transparent manner; and ensuring that research reflects the balance and diversity of priorities within 
populations.’®?” However, with the growing interest and importance of large-scale human genomics 
initiatives worldwide, there has been limited research into how the public are currently being 
involved. There has also been no assessment of the resulting impacts and benefits, including 


genomics initiatives that have involved the public. 


While involving the public in other types of health and medical research has been the subject of 


36,77,78 


previous systematic reviews , comparable reviews have not been published in human genomics. 


Many of the existing reviews on other areas of medical research conclude that reports of 


28.36,79-81 


involvement activities are inconsistent or under-reporte and that the precise ways in which 


people are involved in medical research are not well reported, including any impacts from involving 
people.3#7879 

Our review provides asummary of currently reported public involvement in 96 global human 
genomics projects, listed on a database managed by the Global Alliance for Genomics and Health 


(GA4GH), a recently formed international organisation seeking to enable responsible genomics data 
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sharing within a human rights framework.” The list provides a representation of the current 
landscape of human genomics research worldwide, and a snapshot of contemporary practice with 


regards to public involvement in genomics research. 


This scoping review provides a new perspective by exploring how these genomics initiatives have 
conducted and reported public involvement to date, including any impacts, facilitators and barriers 
of involvement. The intention is that resulting data will help inform future directions for integrating 
public involvement into genomics research and policy development, and inform the development of 


ways of routinely reporting and evaluating any involvement. 


Methods 


Source 


Using a list of human genomics research projects (“initiatives”) from a database hosted by the 
GA4GH (see Supplementary Materials ‘Table 1’), we systematically searched public domain websites 
for information reported on involving the public in research. The database was curated by GA4GH 
staff, last verified August 2016, and contains information about the type of the genomics research 
initiative (i.e. consortium, data-sharing initiative, organisation(s), repository or research project), the 
type of data gathered (i.e. whole-genome or whole-exome sequencing), the geographical scope of 
the initiative, number of participants (cohort size), relevant disease areas, and the public domain 
URL of the website for the organisation or initiative (as some ‘initiatives’ involve a number of 
organisations). The scoping review methodology can be summarized in three stages (see ‘Figure 4.1: 


Scoping review study overview and results summary’ for overview): 


Stage 1 — Defining “involvement” and the search strategy 


We developed a criteria to define ‘involvement’ based on the International Association for Public 
Participation’s participation spectrum and other studies.?9480319.321 This included reports of 
‘consultation’, ‘involvement’, ‘collaboration’ and ‘empowerment’. Involving people in genomics 
research was defined as the ‘active involvement’ in shaping and guiding research, rather than only 
providing data.®°.322.323 We defined specific tasks related to involvement at different stages of the 


research cycle*°° 


, such as the sharing of views to influence research, or co-creating the 
research.7®34325 ‘Consequential’ involvement meant involvement contributing to the research 


process, as distinct from involvement which is ignored or not incorporated.?° 8 We could not 
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always determine whether involvement was consequential based on the available information, so an 


assumption was made that all methods reported resulted were ‘consequential’. 


Stage 2 — Searching websites (data extraction) 


Public domain websites of all the initiatives in the GA4GH database were searched for reports of 
involvement and associated impacts. The date range for website searching and data extraction was 
16th August to 28th November 2017. The exact text from the URLs where data was extracted from 
was collected to allow reanalysis, with all relevant URLs archived using an online archive service to 


preserve the content and the date of extraction.2°° 


We used search engine operators to systematically scan the text of each public domain website for 
relevant phrases, including all grammatical variations of the words used (for example, deriving 
‘involvement’, ‘involves’, ‘involved’ and ‘involving’ from the root word ‘involve’). Grammatical 
variations of specific phrases (denoted by inverted commas) were generated using tables to 
systematically create a series of search strings for each domain. For example, this search string 
returned 4 results: 

site: www.ukbiobank.ac.uk/ “public involvement” OR “involves public” OR “public involved” 


OR “involving the public” OR “involve public” 


Reports of involvement were assessed by a member of the research team (JN), then independently 
assessed by an additional member of the research team, with a random sample assessed by a third 
investigator (PL). Any disagreements between the team on the data included was discussed until a 
consensus was reached. Informed by previous reviews, the search terms for the concept of 
involvement were; “engagement”, “involvement” and “partnership” 222329 3°2 The search terms to 
describe the people involved were; ‘citizen(s)‘, ‘community’, ‘consumer(s)’, ‘lay’, ‘patient(s)’, ‘public’, 


‘stakeholder(s)’ and ‘user(s)’. 


In addition to using a standard list of terms, adaptive (context dependent) search terms were 
sometimes required when searching domains where terms were specific to the region or initiative. 
Adaptive search terms were; ‘advocate(s)’, ‘carer(s)’, ‘civil society’, ‘client(s)?°”, ‘customer(s)?°”, 
‘sroup(s)’, ‘participant(s)’, ‘payer(s)’, ‘population(s)??°’, ‘PPI’ (an acronym commonly used in the UK 


which stands for ‘patient and public involvement’), ‘residents’ (geographical grouping)®’, 
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‘representative(s)’, ‘taxpayer(s)’ and ‘volunteer(s)’. For more details on search method, see 


Supplementary Materials ‘Table 2’. 
Stage 3 — Defining inclusion/exclusion criteria, data synthesis and 
analysis 


Defining the inclusion and exclusion criteria was an iterative process informed by published scoping 
review methodologies.?**3*° Initiatives reporting no involvement were excluded from further 
analysis. Initiatives were categorised as ‘no involvement’ if the context of words such as 
‘participation’ were used to describe ‘research participants’ (research subjects) only, rather than 
aligning with the concept of involvement already articulated.*%3° Reported impacts were excluded 
if they were phrased as anticipated future impacts (using terms such as ‘we expect’), rather than 
reporting real results. Initiatives reporting ‘data sharing' as the only type of involvement were also 
excluded. Initiatives reporting any other type of involvement, according to our definition, were 


included and proceeded to data extraction (structured categorization of extracted search data). 


Extracted data was categorized (data synthesis) based on the following types of information; a) the 
method of involvement (how people were involved)?2?; b) the tasks they were involved in (what 
people did)?”3; c) the stage of the research (using an expanded version of an existing framework’, 
informed by INVOLVE)??’; d) who was involved, for example ‘research participants’, ‘patients’ and 
‘public’ (informed by the Concannon ‘7Ps Framework’ taxonomy)”; e) reported facilitators or 
barriers of involvement; and f) publicly-reported impacts (informed by section 7 and 8 of the GRIPP2 


framework).72323 


As there is currently no standardised way to report and group methods of involving people or 


descriptions of people involved*???78 


, grouping was informed by methods of previous reviews (for 
example, grouping similar methods of involving people???) and by using previously established 
nomenclature.??>331+336 The initial grouping (JN) was reviewed by other authors (PL). While previous 
reviews have used frameworks to label the ‘roles’, ‘degrees’ or ‘levels’ of involvement or 
‘control’*8323, we did not use these frameworks as they require subjective judgements to be made, 


often with insufficient data.325338-340 
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Figure 4.1: Scoping review study overview and results summary 


GA4GH database 
96 genomics initiatives 


Stage 1 — Defining “involvement” 
and the search strategy 


Stage 2 — Searching websites 
(data extraction) 
Systematic and adaptive search of 
public domain websites (n=96) 


Stage 3 — Defining 
inclusion/exclusion criteria, data 
synthesis and analysis Exclude - Initiatives with no 
Data from unique sources (URLs) reported involvement (n=43) 
grouped and analysed. (Number of 
unique data sources = 143) 


Initiatives reporting involvement Exclude - Initiatives reporting 
(n=53) only ‘data sharing’ (n=21) 


Include - Initiatives included in the 
final review (n=32) 
(data synthesis and analysis) 


129 


Results 


Of the 96 initiatives searched, based on our criteria, only a third reported involving people in some 


capacity (32/96, 33%) (Table 4.1). These 32 initiatives were included in the final analysis (data 


synthesis). 


Reported methods of involvement 


The reported methods of involving people were organised into categories, shown below in bold, 


with the number of total initiatives reporting each method shown in brackets: 


Citizen science (n=2) - people involved beyond data collection, research design or data 
analysis, towards co-creation across all aspects of the scientific process”; 
Consultation (n=4) - an organised consultation or dialogue process; 

Formal discussion (n=8) - formalised ‘focus groups’, forums or interview structures; 
Formal groups (n=20) - a working group or committee (including ethics and data access 
committees, ‘scientific advisory groups’ and ‘steering groups’); 

Generic involvement (n=11) - informal, such as meetings, ‘partnership’, or an unspecified 
method; 

Newsletters (n=2) - or mailing lists; 

Online tools (n=7) - websites, social media, or online community hosting; 

Public events (n=13) - with discussion - including initiatives hosting public debates, 
workshops, discussion spaces or conferences; 

Surveys (n=10) - including questionnaires; and 


Other (n=7) - methods not described by other categories. 


Some initiatives reported using multiple methods to involve people. Reports of involving people also 


showed that some methods, for example ‘formal discussion’, can use different modes of 


communication, including face to face, online (for example, ‘massive open online courses’), ora 


combination of the two. 
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Table 4.1: Summary of G44GH initiatives reporting public 


involvement 


Initiatives from a database provided by the Global Alliance for Genomics and Health were searched 


for reports of public involvement (based on public domain websites). Each initiative has been 


assigned an ID number. The type (method) of involvement was categorized using specific criteria. 


Name of Initiative/Organization 


100k Wellness Project 


Australian Genomics Health Alliance 
(AGHA) 
Biobanking and Biomolecular resources 


Research Infrastructure (BBMRI) 
Cancer MoonShot 2020 


Clinical Sequencing Exploratory Research 


(CSER) 


DECIPHER 


East London Genes & Health 


Electronic Medical Records and Genomics 


(eMERGE) 


ELIXIR 


France Genomic Medicine 2025 


Genome in a Bottle 


Genomics England 
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ID 


11 


16 


21 


24 


26 


27 


28 


33 


35 


37 


Geographic Region 
(cohort size) 


North America (100000) 


Australia (1800) 


Europe (N/A) 


North America (20000) 


North America (6000) 


International (21475) 


Europe (100000) 


North America (55028) 


Europe (N/A) 


Europe (N/A) 


International (N/A) 


Europe (100000) 


Reported methods of 
involving people 


Online tools, Other 


Formal groups, Other, 
Public events 

Formal discussion 
formats, Public events 


Generic involvement 


Generic involvement 


Formal groups 


Formal groups, Generic 
involvement 


Surveys 


Consultation, Formal 
groups, Public events 
Consultation, Generic 
involvement 

Generic involvement, 
Public events 
Consultation, Formal 
discussion formats, 
Formal groups, Generic 
involvement, Other, 


Public events, Surveys 


H3Africa 


Implementing Genomics in Practice 


(IGNITE) 


International Rare Diseases Research 


Consortium (IRDiRC) 


Kaiser Permanente Research Program on 
Genes, Environment, and Health (RPGEH) 


Kaviar 


Matchmaker Exchange 


MSSNG 


MyCode Community Health Initiative 


MyGene2 


openSNP 


Precision Medicine Initiative / ‘All of Us’ 


Public Population Project in Genomics and 


Society (P3G) 


Qatar Genome Project 


RD-Connect 
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International (500) 
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International (N/A) 


Asia (1161) 
Europe (2500) 


Formal discussion 
formats, Generic 
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Formal groups, Public 
events 

Formal groups, Generic 
involvement, Other, 
Public events 


Formal groups 


Formal groups 
Formal groups, Online 
tools 


Formal groups 


Formal groups 


Online tools 

Citizen science, Online 
tools, Surveys 

Citizen science, Formal 
groups, Formal discussion 
formats, Generic 
involvement, Online tools, 
Other, Public events, 
Surveys 

Formal groups, Online 
tools, Public events, 
Surveys 

Surveys 

Formal discussion 
formats, Formal groups, 
Generic involvement, 
Newsletters, Online tools, 


Surveys 


The Clinical Genome Resource (ClinGen) 84 North America (N/A) Formal groups, Other 


86 Asia (150000) Formal discussion 
Tohoku Medical Megabank Project formats, Public events, 
Surveys 
Transforming Genetic Medicine Initiative 88 Europe (N/A) Public events 
(TGMI) 
92 Europe (500000) Consultation, Formal 
discussion formats, 
Formal groups, Generic 
UK Biobank 
involvement, Newsletters, 
Other, Public events, 
Surveys 
Undiagnosed Diseases Network (UDN) 94 North America (8000) Formal groups 


96 North America (215000) Formal groups, Public 
Vanderbilt's BioVU 
events 
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Figure 4.2 summarizes overall findings from data synthesis. There was variability in the methods and 


tasks of involvement reported. This supports previous findings that involvement in biomedical 


research is diverse, varied and described using different language.**? 


Figure 4.2: Summary of global genomics review results 
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Conclusions 


e Involving people in future genomics research is essential to maintain public trust, improve research 
and ensure that access to the benefits of research is equitable 

e The method and tasks of involving people are varied 

e The evidence for effective ways of involving people is not clear with data likely under-reported 

e Consistent reporting and evaluation of involving people is required 

e This review may assist informed decisions about planning future involvement in genomics research 


Reported tasks of involvement 


The tasks people were involved in (what people did when they were involved) were diverse. Tasks 
included; identifying research priorities related to people with specific diseases; communicating 
priorities to scientists, clinicians and health policy makers, [IDs 11, 37, 50, 74]; designing or 
improving how people will be involved in the research [IDs 41, 50]; educating professionals involved 
in the research [ID 8]; developing workshops and conferences [IDs 44, 94], offering culturally 
appropriate information about research to people in community groups [ID 37]; providing feedback 
on the cultural and linguistic appropriateness of public domain research documents [ID 96]; and 
translating information into 'lay' language [ID 92]. Tasks also involved sharing views and perspectives 
about multiple aspects of research projects [ID 37, 92, 96]; articulating phenotypes [ID 65]; and 
being a project co-investigator [ID 21]. 

Some initiatives reported involving people in the task of giving feedback and sharing views and 
perspectives about the ‘acceptability’ of specific aspects of the research design. For example, 
research management, governance [IDs 27, 41, 92, 72], accountability, planning, policy, protocols, 
data access and data use [IDs 37, 74, 84, 92], consent, re-contact, withdrawal, confidentiality, 
benefit sharing, project closure and recruitment [IDs 37, 62, 74, 92]. Anumber of initiatives also 
involved people in the task of sharing views and perspectives on issues of perceived social and 
ethical importance (including being told about potentially serious incidental findings) [IDs 37, 74, 


96], or to scrutinise a project to ensure it aligned with public interest [ID 11]. 
Reported stages of involvement 


Most reports of involvement were at the ‘implementation and management’ stage of research 
(19/32, 59%), followed by ‘dissemination’ (12/32, 38%), ‘evaluation’ and ‘study design’ (both 9/32, 
28%) and ‘data analysis’ (8/32, 25%). The stage with the lowest number of initiatives reporting 
involvement was ‘funding’ (1/32, 3%) with the next lowest being ‘identifying topics’ and 
‘prioritisation’ (both 4/32, 13%). Four initiatives reported involving people at every stage of research 


[IDs 21, 50, 69, 74]. 
Reported impacts of involvement 


Nearly 10% of the initiatives reporting involvement also reported impacts of the involvement (3/32, 
9.4%). The method with the most reported impacts was ‘public events’ (4/8, 50%), followed jointly 
by ‘formal discussion formats’ and ‘surveys’ (2/8, 25%). Actions taken as a result of involving people 
included the creation of a mobile outreach bus [ID 37]; improvements to ethical and governance 


frameworks [ID 92]; and improved participant information and consent documents [ID 37]. Three 
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initiatives reported a total of eight distinct impacts as a direct result of involving people. Some 


impacts were reported as being a result of using a combination of methods. 
Reported facilitators and barriers to involvement 


A number of specific facilitators of involvement were reported, including: reimbursement policies [ID 
21], with people involved paid for their time [IDs 92, 94], travel [IDs 74, 94] accommodation [ID 74] 
and expenses [IDs 74, 92]; education and learning opportunities for the general public [IDs 1, 11, 41], 
ensuring people involved are informed and can make informed decisions [ID 11]; education for 
health professionals [IDs 41, 50], providing opportunities to learn about how to involve people [IDs 
41, 50], and governance which is trusted by all stakeholders to be able to manage real or perceived 
competing or conflicting interests [ID 50]. The only barrier reported was limited venue size, which 
inhibited the number of people involved [ID 92]. This also implies a limited budget, which is an 


important but likely under-reported implicit limitation on all involvement methods. 
Discussion 


This review provides an overview of reported public involvement occurring in prominent human 
genomics projects worldwide, during a period of rapid growth for genomics research. We identified 
significant variability in the way in which involvement is reported. This variation of reported 
involvement suggests diversity in both the ways people are being involved in genomics, and in the 
varied and emergent language used to report and describe involvement, consistent with other areas 
of biomedical research. 3°34? While there are similarities with the principles of involvement in other 
kinds of biomedical research, this review has identified three different tasks not found in other 
reviews.’ 

Because the results from this review suggest there is currently no standardized way of reporting and 
therefore evaluating how people are involved, there is a risk that best-practice will be hard to define 
or even absent in future evidence reviews. This has implications, as the number of people involved in 
genomic research is predicted to grow exponentially. Without a standardized framework to report 
and transparently evaluate ways people are involved, it will be difficult to create an evidence base to 


inform best-practice. 


While a third of initiatives reported involvement, a majority of projects did not (64/96, 66%). Some 
prominent initiatives involving the genetic analysis of thousands of people did not refer to public 
involvement in any way. This is somewhat concerning given that involving the public has been 


identified as a crucial aspect of responsible research practice in genomics.?? Whilst we acknowledge 
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the probable under-reporting of involvement activities on public-domain websites, we argue public 


involvement in human genomics needs to increase. 


Findings from this review also suggest it is best-practice to involve multiple stakeholders (including 
the public) in designing how people will be involved in research (co-design of involvement plan), to 
involve the public throughout the lifetime of a project in certain tasks (such as overseeing data 


access) and to evaluate the involvement with both qualitive and quantitative data. 


Involving people in planning involvement may improve how appropriate, effective, efficient and 
equitable it is. Involving people in the design of planned methods of involvement by identifying what 
is considered ‘good practice’ was reported by H3Africa [ID 41] and the International Rare Diseases 
Research Consortium (IRDiRC), and reported as a facilitator of involvement by the IRDiRC [ID 50]. The 
IRDiRC [ID 50] also reported both qualitative and quantitative data should be used to evaluate 
involvement, although there is currently no way to systematically collect and analyse such 


activity.126 


If involvement is more effective when the public are invited to help plan it, standardised reporting 
and evaluation will help make informed decisions at every stage of involvement from co-design 


through to evaluation. 
Implications for policy and practice 


With the impact of some genomics research data likely to be measured in decades, some of the 
initiatives offer a useful insight into planning and funding sustainable (long-term) involvement for 
the entirety of an initiative’s lifetime.®’ For example, Genomics England [ID 37] and the UK Biobank 
[ID92], as exemplars, both reported multiple ways of involving people, at different stages of the 
research cycle, conducted over a number of years. Other initiatives, such as the International Rare 
Diseases Research Consortium (IRDiRC) [ID 50] and the Public Population Project in Genomics and 
Society (P3G) [ID 72], also publicly stated the importance of planning sustainable involvement over 
the duration of a project. These initiatives demonstrate a standard of involving people which could 


eventually be used to inform international best practice. 
The IRDiRC also reported that involving people throughout an entire project helped maintain trust 
by scrutinising and managing competing or conflicting interests [ID 50]. Similarly, the UK Biobank 


reported that involving people in the ethics and governance should not be one-off and must be 
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ongoing [ID 92]. The method of using ‘formal groups’ was more common for more complex or 
ongoing tasks such as overseeing data access, policy development, research management and 
improving research protocols. 

Some initiatives, such as openSNP, reported tasks that were specific to genomics research, such as 
articulating phenotypes [ID 65]. Involvement in this kind of task might have important implications 
when working to usefully describe people’s subjective lived-experiences across multiple languages, 


for example, rare diseases and mental health.?® 


Public involvement in articulating phenotypes also suggests that the traditional boundaries between 
terms such as ‘research’, ‘healthcare’, ‘patients’, ‘research participants’ and ‘the public’ may be 
increasingly challenged by the methodology of future genomic research.*“ Findings from this review 
show that both ‘the public’ and ‘patients’ are already involved in every stage of research, including 
collecting and analysing data.* Any future standardised reporting of involvement will need to keep 
pace with the continually evolving language to describe not only what research is, but who is 


involved and how. 


Limitations 


While the database hosted by GA4GH includes many of the most prominent human genomics 
research initiatives worldwide, the database is not exhaustive. There are several known genomics 
initiatives which involved people that were not part of the database. Therefore, the GA4GH selection 
cannot be considered entirely systematic or representative. However, it does provide a reasonable 


indication and snapshot of the current global landscape in human genomics research. 


Our data collection was limited to self-reported information reported on English language websites 
only. This likely under-reports the total amount of public involvement occurring. For example, some 
initiatives may have conducted involvement, and not reported it publicly. This indicates a current 
lack of standardization or best-practice in reporting involvement activities in human genomics 


research, which we feel could be improved. 


Of the public involvement activities reported, we did not systematically follow up reports to confirm 
they had taken place, or if involvement was ‘consequential’ .3?®-328 While this is a limitation of the 
review, it also reflects the inconsistent and often incomplete ways genomics research initiatives 


report impacts of involving people. For example, the impact of how involvement influenced research 


138 


was only reported by three projects - Genomics England [ID 37], the Qatar Genome Project [ID 73] 
and the UK Biobank [ID 92]. 


A number of reported methods did not provide sufficient information to make a clear decision about 
how to group a method. For example, many reports of involvement simply referred to a ‘workshop’, 
‘meeting’, or other ‘public events’, where people were able to get involved by sharing views and 

perspectives. As a result there is potentially significant overlap between some methods, which could 


have been articulated more clearly if more data were available. 


Reports of ‘data sharing’ were excluded, as they were not considered as public involvement. While 
sharing data may enable people to be involved in some capacities (for example, in analysing data), 
data sharing is not necessarily an indicator that people were involved in the analysis of data. The 
complexity within the term ‘data sharing’ in genomics, and how people can be involved in the 


analysis and interpretation of data also requires further consideration.119:345.346 
Conclusion 


Involving people in the future of genomics research is essential to maintain public trust, improve 
research outcomes, and to ensure that access to the benefits of research is equitable.2°7®?"4 While a 
third of initiatives reported involving people, only 10% of initiatives reported impacts. The limited 
reporting of involvement suggests there would be intrinsic value in developing a more systematic 
method of both reporting and evaluating how people are involved in human genomics research. 
Data from such reporting could provide the evidence required to inform future policy around 


involvement of the public, as human genomics research continues to grow. 


Supplementary Material 


1. ‘GA4GH database and results table’: This table combines a database of human genomics 
research projects hosted by the ‘Global Alliance for Genomics and Health’ with a summary of 
results from a scoping review of currently reported public involvement reported on public 
domain websites. The data is organised alphabetically, with organisations reporting 
involvement listed first. A direct link to Additional File 1 can be found here: 


https://www.frontiersin.org/articles/file/downloadfile/446268 supplementary- 


materials tables 1 xlsx/octet-stream/Table%201.XLSX/1/446268 


139 


2. ‘Systematic search method’: This document describes the search method, including how 
standard and adaptive were used to search the public domain websites of all the included 
initiatives in the GA4GH database for reports of involvement and any impacts. A direct link 
to Additional File 2 can be found here: 


https://www.frontiersin.org/articles/file/downloadfile/446268 supplementary- 


materials tables 2 docx/octet-stream/Table%202.docx/4/446268 


This file is also included in the Appendices. 


This is the end of the chapter adapted from the published, peer-reviewed 2019 review titled ‘Public 
involvement in global genomics research: A scoping review’ at this link: 


https://doi.org/10.3389/foubh.2019.00079 
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Chapter 5 — Involving elderly research 
participants in the co-design of a 


future multi-generational cohort study 


This chapter is adapted from the published, peer-reviewed 2021 article titled ‘Involving elderly 


research participants in the co-design of a future multi-generational cohort study’.® 


Please note: 
e Figure numbers have been changed to align with this thesis, and some formatting has been 
adapted, including integrating references into this thesis. 
e This paper uses the term ‘enablers’, to avoid confusion with those facilitating discussions, 
where previous chapters used the term ‘facilitators’. 
e The published version can be found here, along with the additional files: 


https://doi.org/10.1186/s40900-021-00271-4 
Authors 


Jack Nunn, Merrin Sulovski, Jane Tiller, Bruce Holloway, Darshini Ayton, Paul Lacaze 
Plain English Summary 


Research which follows a population of people over time is a valuable way to learn about what 
contributes to both wellness and disease. Some studies of this kind look at multiple generations and 
such studies have had significant positive impacts on public health. However, such studies are 


challenging to establish and expensive to maintain. 


It has been proposed that the existing ASPirin in Reducing Events in the Elderly Extension study 
(ASPREE-XT) would be a good basis for a future multigenerational research study (MGRS). There is 
evidence that involving potential participants in co-designing research can improve the quality of the 
study design, recruitment and acceptability of the research. A team of people including a current 
ASPREE-XT participant were involved in planning how potential participants would be involved in the 


co-design process. An advert was sent to 14,268 participants of the ASPREE-XT observational cohort 
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study to invite them to be involved in the process of planning a future multi-generational research 


study. 


Fifty-nine ASPREE-XT participants were interviewed by telephone and 18 attended a face-to-face 
workshop event. We used a newly developed standardised format to plan and report how 
participant involvement activities positively impacted the study design (Standardised Data on 


Initiatives - Alpha Version 0.1). 


Involving participants positively impacted the proposed study design by improving the research 
objectives, developing protocols, influencing funding decisions and improving ethics applications. 
This case study provides evidence that including participants in the design of a research study 


positively impacted the study design, participants and researchers. 
Abstract 


Objectives: To report as a case study the process of involving study participants and how this 
impacted the co-design of a proposed multi-generational research study, using a novel standardised 


reporting tool. 


Background: An advert was sent to 14,268 participants of the ASPREE-XT observational cohort study 
to invite them to be involved in co-design and planning of a future multi-generational research 
study. Participants were aged 74 years and older from Australia, located in both metropolitan and 


rural locations. 


Interventions: We used participatory action research to involve elderly research participants in the 
co-design of a proposed multi-generational cohort study between 2017-2019 using newsletters, 
telephone interviews and an in-person workshop. Primary and secondary outcome measures: We 
used the ‘Standardised Data on Initiatives (STARDIT)’ Alpha Version to plan and report how 


participant involvement activities positively impacted the study design. 


Results: Fifty-nine ASPREE participants were interviewed by telephone and 18 participants attended 
a face-to-face workshop event. Involving participants positively impacted the proposed study design 
by improving the research objectives, developing protocols, influencing funding decisions and 
improving ethics applications. Learning points included the importance of maintaining the ideals of 


the ASPREE study (respect, quality and transparency); research participants’ preference for the 
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option of receiving results (including genetic results); participants’ need for involvement in decisions 
about recruitment, data access, governance and other ethical issues; and the preference for 
different communication methods, including both face-to-face and online methods. The process was 
highly valued by all stakeholders, including research participants, study staff and lead investigators. 
Involvement of participants was described by the lead study investigator as “enormously helpful”. 
Conclusions: This case study provides evidence that including participants in the design of a research 
study positively impacted the study design, participants and researchers. Using a standardised 
reporting tool to describe the methods and impacts provides a way for learning from this case study 


to inform future research studies planning to involve people. 
Introduction 


The ASPirin in Reducing Events in the Elderly (ASPREE) trial (2010-2018), a randomised controlled 
trial for aspirin in healthy older people, recruited 19,114 participants from Australia and the USA47 
350 The median age at recruitment was 74 years. The ASPREE Healthy Ageing Biobank is a sub-study 
which has collected biospecimens from over 15,000 ASPREE participants, alongside detailed medical 
records, lifestyle, cognitive function and physical testing data. 3t ASPREE researchers are conducting 
various types of genomics and biomarker research. ASPREE had a remarkably high retention rate, 
with only 1.2% of participants withdrawing from the study and 90% still attending annual visits after 


an average of 4.5 years follow-up.2927°3 


ASPREE XT is a follow up observational study that was established in 2018 to continue to collect data 
from ASPREE participants for another 5 years. ASPREE-XT participants are over the age of 74 
(defined as elderly?) and Australian participants are located in both metropolitan and rural 
locations. Multiple stakeholders, including a participant advisor, proposed a new multi-generational 
research study (MGRS) which could recruit direct descendants of ASPREE-XT participants, as part of a 
longitudinal observational study alongside their relatives for two generations or more. MGRS are 
challenging to establish and expensive to maintain, yet their value to medical and epidemiological 
research is significant. ASPREE-XT has been proposed as a platform for a multi-generational study. 
Recruiting ASPREE-XT participants and their descendants to a MGRS would provide a large starting 


population by international standards. 


Previous multi-generational, longitudinal cohort studies have had significant positive impacts on 


public health. Examples include the Framingham Heart Study,?°° Lothian Birth Cohort?°® and Dubbo 
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Osteoporosis Epidemiology Study,?°” 


all of which had substantially lower starting populations than 
ASPREE-XT. 38 ASPREE-XT provides a rare opportunity for such a study in Australia, already 
combining high-quality medical record data with genomic data on a large number of elderly 
Australians. %8 Additionally, the cohort is already well engaged, with all surveyed participants 


supportive of a proposed MGRS. 


The aim of this study was to report as a case study the processes of involving ASPREE-XT participants 
in the co-design of a proposed multi-generational research study, and how this impacted study 
design. The research objectives were to involve potential participants in the co-design of a new 
MGRS; plan the process in a standardised way; identify themes and preferences; and evaluate then 


report the process using a standardised reporting tool.*°* 


ASPREE-XT’s unique focus on healthy ageing adds substantial value to public health, medical, 
epidemiological and geriatric research. Previously collected genomic (including epigenetic) data, 
combined with detailed medical data and ongoing cognitive assessment, allow that a MGRS with the 
ASPREE-XT cohort would be of considerable value to science for decades to come, for example in 
dementia research.*°? The proposed study would examine health outcomes in a large, well- 
monitored cohort; and provide data to help inform our understanding about the genetic and 
environmental determinants of health and disease, across multiple generations. However, this is 


only possible if people choose to participate. 


Many clinical research studies are underpowered due to poor recruitment and retention?®. Involving 
participants and the public in research design has been shown to improve the recruitment 703°, 
quality and relevance of research. ?”362 The concept of involvement is research being done ‘with’ 
people rather than ‘on’ them. 2°° Involving the public, patients, research participants and other 
stakeholders in actively contributing to the research process can lead to a range of positive impacts 


and outcomes. 279 


In human genomics research, the need to involve the public and other stakeholders is a crucial 
aspect of responsible research practice. 2°?”41 The term ‘stakeholder’ here means anyone who has a 
‘stake’ in the research, in particular those with important knowledge, expertise or views that should 
be taken into account. 7°° This includes ASPREE-XT participants, study staff and academic research 
investigators and the wider public. At the earliest stage of the research cycle (the conceptual stage), 


some current ASPREE-XT participants were invited to be involved in the co-design of anew MGRS. In 
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this paper we aim to outline how people were involved in the co-design process in order to appraise 


the methodology and inform future best-practice. 
Terminology 


We have used consistent language in this paper to describe concepts such as ‘involvement’. To aid 


readers, Table 5.1 provides definitions of important terms used consistently throughout this paper. 
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Table 5.1: Definitions of terms 


Involvement — The words ‘involvement’ or ‘being involved’ describe the concept of people being 
‘involved’ in research. This is when research is carried out ‘with’ people rather than ‘on’ them. >° 
‘Involvement’ can also be defined as when other people aside from the research team, such as the 
public, patients, research participants and other stakeholders, actively contribute to the research 
process. °?” It is the ‘active involvement’ in shaping and guiding research, rather than only 
providing data. 8%322323 

Participant — a person who participated in the process of sharing views and perspectives about 
the proposed MGRS, including sharing views about preferences for any future involvement. The 
term ‘ASPREE-XT participants’ will be used when specifically referring to participants from the 
existing study. 

Participant advisor — before inviting people to become participants, it was necessary to involve a 
small number of participants to help advise and plan the process. These participants were chosen 
from an existing reference group of ASPREE-XT participants. 

Stakeholder — this term includes anyone who has a ‘stake’ in the research, in particular those who 
have important knowledge, views or perspectives that should be taken into account. 7°%° In this 
paper it refers to participants, participant advisors and ASPREE-XT study team members (including 
researchers, ASPREE-XT participant assessors and lead investigators). 

Study team — this process was guided by the study team, who consisted of academic researchers, 
ASPREE-XT participant assessors, a participant advisor and a lead investigator. 

The process — this term will be used to describe the process of involving ASPREE-XT participants 
by inviting them to share views and perspectives about a potential future MGRS. This process 
includes the co-creation of this case study with participant advisors. 

Participatory action research (PAR) - this term describes a number of related approaches, 
including forms of action research which embrace a participatory philosophy and include ‘co- 


design’ and ‘co-production’ of research. 849 


146 


Materials and Methods 


Study design 


A participatory action research (PAR) method was chosen to guide the process with co-design and 
reporting guided by a number of frameworks. 7287123363364 Participatory action research is an 
umbrella term which describes a number of related approaches, including forms of action research 
which embrace a participatory philosophy and include ‘co-design’ and ‘co-production’ of research. 
841) During the study design, we applied this co-design process where researchers and other 
relevant stakeholders (including research participants) “work together, sharing power and 
responsibility from the start to the end of the project”,®° including knowledge generation and 


translation. 83t Figure 5.1 summarises the process we used (Fig 1). The ASPREE participant advisor 


was an integral member of the study team, through each stage. 
Guiding Frameworks 


The process was guided by a number of international participatory action research methodology 
frameworks?®+365:36 including the International Collaboration for Participatory Health Research.??° 
The participatory action research method was also informed by an international review of 
involvement in genomics research carried out by some members of the study team”. Learning from 
this review informed the subsequent development and application of an Alpha version of 
‘Standardised Data on Initiatives’ (STARDIT).*°? STARDIT includes a tool to map people’s preferences 
for involvement in a standardised way, including mapping views on who should be involved and 
how. STARDIT was then used to guide co-design of the process, and to subsequently report how 
people were involved, using standardised data. An Alpha version of the STARDIT framework was 
also used in parallel with the thematic analysis to organise data into pre-defined ‘super-categories’ 


which allow consistent comparison with other data using this reporting framework. 1°? 


We used a case study research methodology to record and describe the process of involving 
participants in the co-design of a proposed MGRS. Case study research methodology is a form of 
empirical inquiry and this study is presented as an instrumental case study, where the purpose is to 
understand the particular case and to provide data that could produce useful generalisations by 
using inferences from the data. 15%(P109) 

The selection of the population for this case study was informed by a number of factors which were 


appraised by the study team, including ethical, pragmatic and population considerations.1”*°” One 
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of the study team members (PL) is an investigator with the ASPREE-XT study, which was used as a 


starting point by the study team to explore the appropriateness of this case study. 
Data collection and reporting 


The data collection and reporting was informed by a number of frameworks for reporting 


involvement in research.2°7987179- 


181,192,368 Qualitative and quantitative data was collected from 
multiple sources, including meeting notes, audio recordings, documents, emails and surveys. data 
sources were analysed using a number of methods including thematic analysis. Further information 


about how data sources were analysed is in Table 5.2. 
Stages of research 


Figure 5.1: Process timeline of the participant involvement 
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Stage 1: Planning 


The study team held four meetings to co-design the involvement activities. One participant advisor 
was involved in a number of tasks including reviewing and improving the written information, 


telephone interview questions, and the facilitation plan for the event. 
Stage 2: Recruitment and telephone interviews 


An advert was placed in a newsletter to 14,268 ASPREE participants. The process we used for 
recruiting and involving ASPREE-XT participants has been divided into four chronological stages, 


summarised above in Figure 5.1. 


After the recruitment and consent process, participants were interviewed by telephone. A summary 
of the questions and responses from the telephone interviews is shown in Table 5.3. For a list of all 


questions asked, see Additional File 2 in the Appendices. Participants were asked about their 
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willingness to provide feedback throughout the study, and to be involved in study design, as well as 
preferences for modes of communication. The definition of involvement below used in the script 
was co-designed with participants for subsequent interviewees, with versions 1 through 1.3 piloted 


with 15 participants. 


Pre-question script: ‘Traditionally, research studies have been designed and conducted only by 
researchers, and people invited to participate. We are challenging this idea of researchers being the 
only experts.’ 

Question: Would you be willing to provide feedback and share your views and perspectives by 


commenting throughout the research process?’ 


Twenty relevant interviews were transcribed, coded and categorised (JN), with relevant interviews 
identified by two investigators independently (JN, PL). 18 To reduce any unconscious selection bias, a 


sample of over 10% of the interviews was selected at random. 
Stage 3: Event 


Eighteen participants attended a four-hour workshop event in central Melbourne, led by JN, who is 
an experienced facilitator. The event was co-designed by the study team, and was informed by 


interview data and international best-practices for involvement events. 38636? 


The event included an introduction to the proposed MGRS by the lead ASPREE-XT genomics 
researcher (PL); a plain-English introduction to genomics by an expert in genomics who is also an 
ASPREE-XT participant (BH); a summary of the telephone interview results by the interviewer (MS); 
and an interactive session which included open questions about the types of information 


participants would like returned and recruitment of family members (JN). 


The final session included a presentation and interactive discussion about involvement in research, 
led by the event facilitator (JN). This session explored preferences about how people would like to 
be involved, with open and closed questions. Questions included preferences about tasks and modes 


of communication. 


Throughout the event, participants shared their views on a range of issues through interactive 


discussions, voting (by show of hands) and anonymous written feedback. 


Stage 4: Evaluation and analysis 
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Table 5.2. Summary of data collection and analysis 


Data source description Analysis method 
Meetings — including meeting notes, recordings and relevant Qualitative (content analysis) 
documents 
Telephone interviews — including recordings, notes made by Qualitative (thematic analysis, 
interviewer, summary documents and related emails content analysis) 

Quantitative (number of responses) 
Event — including audio and video recordings, interviews, Qualitative (content analysis) 
written notes and feedback forms Quantitative (number of responses) 
Study team surveys — responses to open ended questions by Qualitative (content analysis) 
email 
Other data — this included reflexive research diaries, relevant Qualitative (thematic analysis) 


emails, financial and other relevant documents Quantitative (analysis of cost) 
Evaluation and reflection 


All members of the study team (except JN) were surveyed six months after the face-to-face event in 
order to integrate the valuable views and perspectives of those involved in co-designing and 
delivering the process. Design of surveys was informed by frameworks for planning and reporting 
public involvement.”?*” The study team were asked 11 questions (available in Additional File 1 in the 


Appendices) and the data from the four interviews was coded and categorised (JN) using STARDIT.1°2 
Data analysis 


The stages of qualitative data analysis included data mapping and familiarisation; transcription; 
coding; searching for themes; reviewing themes with study team members; labelling and 
summarising themes; and reporting the findings.*”* In order to enhance validity of the analysis, two 
authors (JN, MS) independently analysed the data thematically, which was then checked 
(triangulated) by a third author (PL).14917837° Standardised categories (STARDIT) were used during 
content analysis of the data in order to facilitate comparison with other research projects.’? More 


information about the data sources and a STARDIT report available is in Additional file 1 in the 


Appendices. 
Results 


The results are presented in three sections. Section one provides results from the stages outlined in 
Figure 5.1, section two shares the perspectives of ASPREE-XT participants and other stakeholders, 


and in section three we describe how participant involvement impacted study design in seven ways. 
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We quote participants directly, assigning each a unique number. Figure 5.2 summarises the entire 


process, results and impacts. For quantitative data from the telephone interviews, see Additional 


File 3 in the Appendices. 
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Section 1: Results from stages 1-4 


Results from Stage 1: Planning 


The input of the participant advisor during the early co-design stage had clear benefits, in particular 
in identifying the best approaches for engaging the broader ASPREE population; improving the 


wording of participant information resources and improving question design for the interviews. 
Results from Stage 2: Recruitment and telephone interviews 


After reading the advert in the newsletter, 76 ASPREE-XT participants agreed to participate in the 
process. We interviewed 59 people by telephone, with the remaining 17 not responding to a number 


of follow-up calls. 


Interview participants expressed a strong interest in receiving results from genetic research. The 
strongest preference was for genetic results of medical significance (rather than ancestry or drug- 


response). 


All telephone interview participants expressed willingness to provide feedback throughout the 
process of planning the proposed MGRS. While most participants understood the concept of 
involvement, 9 of the 20 analysed expressed self-doubt about how they could be involved. Of 20 
analysed telephone interviews, five participants seemed unclear about what tasks they could be 
involved in, suggesting the concept of participant involvement was new to them. Six participants 
sought clarity about what was expected of them when being involved, or discussed the skills and 
knowledge required. Further, participants stated that the goals and processes of involvement 
needed to be clear in order to avoid “spending too much time” and were willing to be involved as 


long as the task had purpose and was not “just for the sake of chatting” (P1). 


During the telephone interview stage, the interviewer initially “struggled to explain” the concept of 
involvement (as distinct from participation in an active study) (MS). Co-designing and making 
changes to the language used in the telephone interviews appeared to improve participant 
understanding of the concept of involvement. Changes included providing a clear definition and pre- 
question script and including actual examples of involvement (such as overseeing ethical decisions 


about data access), thereby avoiding jargonistic abstract concepts such as ‘involvement’. 
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Table 5.3: Summary of data from telephone interviews 


Questions (closed) 


1. Do you think participants like yourself should be 
involved in helping design research projects, or just 


researchers? 


2. Would you be willing to provide feedback and share 
your views and perspectives by commenting 
throughout the research process? 

3. Would you like to be more involved in study design 


rather than just participating? 


4. Would you be more or less likely to participate in 


research if participants were involved in design 


5. What is your preferred mode of communication for 


being involved 


6. If genetic research results were made available, 
which types of genetic testing would you be 


interested in? * 


7. If only one option for genetic testing was available, 


which one would you prefer? 


Results 


Just researchers 
Participants involved 


Unsure 


Yes 


No 

Yes 

No 

Unsure 

More likely 

Less likely 

Unsure 

Wouldn’t influence 
participation 

Face to face event 
Post 

Online discussion 
Online questionnaire 
Medical 

Ancestry 

Drug response 
Medical 

Ancestry 

Drug response 


No preference 


% of interview 
participants 
39 

46 

11 


18 


*Note that percentages indicate the number of participants who responded with that response. On 


question six, participants could select multiple answers, so totals do not add up to 100%. 
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Results from Stage 3: Event 


Eighteen of the invited 76 participants attended the face-to-face event. When specific tasks were 
discussed at the event, 10 participants expressed interest in being involved in recruitment and 
communication tasks, 7 in data access decisions, and 11 in ethical decisions. When surveyed after 
the event, the study team reported that involving participants positively impacted the proposed 
study design by improving the research objectives, developing protocols, influencing funding 
decisions and improving ethics applications. Videos of the event discussions and interviews with 


participants will be shared in the public domain. 
Results from Stage 4: Evaluation and reflection 


Participant feedback from the workshop event was analysed, with participants reporting it as a 
positive experience, one describing it as “brilliant” (P7). When surveyed, all members of the study 
team reported that the event achieved its intended aims and was a positive experience. No negative 


impacts were reported from any participants or study team members at any stage of the process. 


One study team member described that through involving people, their perceptions 
“significantly changed” as they learned how participants could have “valid, interesting, and often 


novel ideas that researchers may not have considered”. The lead investigator (PL) noted: 


“What | learnt is that sometimes the researchers’ assumptions about certain things 
can be off or even incorrect, and that researchers can miss critical points that are important to 


participants and the study”. 


During the process, both participants and study team members reported changed views about the 
value of involvement in research, demonstrating ‘transformative learning’ and co-construction of 


knowledge. 12127263 
A number of significant learning points were identified by the study team when responding to the 


question “do you have any advice to other researchers planning involvement for their research”. 


Significant learning points are summarised in Table 5.4. 
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Table 5.4: Summary of learning points from study team 


Summary of learning points from study team 

1. Fund and prioritise involvement, make it a requirement 
2. Ethics processes take time, but can improve plans 

3. Know your audience — don’t make assumptions 

4. Value diversity in experience and knowledge 


5. A supportive team improves the experience for all 
Cost and value 


The entire process of involving people was estimated to cost $10,000 AUD, including staff time, 
catering and event venue hire. The value of the process was summarised by the lead investigator 


who stated “I learnt a lot from the process and am very glad we made the effort”. 
Section 2: Perspectives of ASPREE participants and other stakeholders 


Recurring themes were identified by the study team from the data sources analysed, and seven 
specific impacts were reported as a result of involving participants (summarised below). The analysis 
is divided into two sections: (1) participant views about the proposed study, (2) stakeholder views 


about involvement in the proposed study. 
1: Participant views about the proposed study 


Participants were very positive about the proposed MGRS, with one stating ‘I think it sounds very 
good’ [P20]. Altruism was a primary motivator for participation, with participants suggesting that 
outcomes of a MGRS could include benefits to themselves (personally and for their families); 
improving healthcare more generally; and the potential for saving lives, preventing diseases, 


improving quality of life, and improving future research. 


Participant views on types of information they would like returned from genomics research and how 
the study should be funded (commercial funding versus public funding) were diverse and did not 
always align with the study team’s prior expectations. For example, two thirds of event participants 
wanted access to their own genomic data, which was described by the lead investigator as ‘very at 


odds with the current system’ (PL). 
2: Stakeholder views about participant involvement in the proposed study 


Participants were supportive about being involved, with all participants supportive of being involved 


by providing feedback throughout the research process (100%, 32/32), with a typical participant 
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response being “d be happy to be involved’ [P3]. Views about enablers were shared in three of the 
20 interviews coded, by all 18 of the event participants and all study team members surveyed. Views 
about barriers were shared in eight of the interviews coded and by half of the study team surveys. 
Further data is categorised in Table 5.5. Additional mapping of preferences for involvement using 


the STARDIT-PM tool can be found in Additional File 1. 1% 
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Table 5.5: Enablers and barriers for involving participants 


Enablers 

Financial remuneration for people’s time; 
financial support for travel and 
accommodation 

Learning and development opportunities 


for participants and researchers 


Small groups at events gave more people 


a chance to share perspectives 


Early notice of events 


Clear information about timings and time 
commitments, frequency of involvement 
and available support 

Clear information about purpose and 
expectations of involvement, feeling their 


involvement has consequences 
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Quotations 
Running a business ‘limits me and 


my time’ [P15] 


‘if you tell me what would be useful, 


| could do it’ [P13] 


‘Small group’ discussions at the 
event ‘worked well’ (Study team 
member) 


‘give me enough notice’ [P10] 


Ensure ‘people are advised what’s 
going to happen at the workshop’ 
[P8] 

‘What’s the endpoint — what’s the 


goal?’ [P11] 


Barriers 
Living in rural areas and travel 
logistics a barrier to participation in 


face-to-face events 


Poor explanation of abstract concepts 


such as ‘involvement’, which can be 


Jargonistic 


Self-doubt about their skills or 
knowledge mean they don’t think 
they could be involved 

Some people not comfortable being 
part of a face-to-face group 

Lack of clarity about expected time 


commitments 


Unclear about what tasks they could 


be involved in 


Quotations 

‘Td like to be involved online 
rather than face to face because 
of travel difficulties’ [P6] 

‘there seemed to be confusion, or 
a lack of understanding of what 
this ‘involvement’ would actually 
look like’ (MS) 

‘ld probably ask a stupid 
question’ [P2] 


‘| get very uncomfortable ina 
group of people’ [P12] 
Being involved ‘depends on 


what’s involved and time’ [P1] 


‘I don’t know how | could but 


willing to help’ [P13] 


Enablers 
Independent facilitator when working in 


groups (either face-to-face or online) 


Short events ensure people do not get 


fatigued 


Having access and literacy in using 


computers and online tools 


A selection of flexible communication 
modes (such as face to face and 


facilitated online discussion forums) 


Involving people in research ethics and 


governance 
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Quotations 
I’d feel more comfortable if | had 


someone who was facilitating [P14] 


‘any longer and | think fatigue would 
have dampened the enthusiasm’ 
(Study team member) 

‘if | could negotiate [online 
discussions] l’d be happy to do that’ 
[P15] 


‘d be happy to be involved — more 
online but if there was an occasional 
need to come into the city I’d be 
happy to do that’ [P3] 

‘ethics is the difference between 
right and wrong — you know what’s 
right and you don’t do what’s 


wrong’ [P18] 


Barriers 
Face to face discussions ‘dominated’ 
by more confident or knowledgeable 


people (Study team member) 


People feel they have limited time, 
are busy with work or social 
commitments 

Lacking access, literacy or trust in 
using computers, smartphones or 


online tools (including social media) 


Only one mode of communication, 
such as expecting people to travel to 


events 


A ‘researchers know best’ attitude 
that doesn’t value the process of 
involving people (Study team 


member) 


Quotations 

‘Participants from professional 
backgrounds to some extent 
dominated some of the 
discussions’ (Study team member) 
‘| don’t have a lot of time left in 


life’ [P1] 


‘m hopeless with computers’ 


[P16], 1 don’t have internet’ [P12] 


‘online is often easier’, face to 
face only ‘as long as it’s not too 
far’ [P22], ‘travel distance is an 
issue’ [P20] 

Researchers ‘don’t see the forest 


for the trees’ [P19] 


The lead investigator stated “the feedback has been so valuable” and that it will be “built into the 
design” of future research. However, the study team also identified barriers to involving people 
which exist for researchers. One study team member reported that at the start of the project they 
were “unsure how people without a science or health background” could be involved. They reported 
a personal shift during the involvement process from not understanding how participants could be 
involved and being concerned about “asking too much” to believing that, “with adequate resources 


(financial, training, time...), people can be involved in all aspects of genomics research”. 


Other barriers identified by the study team included delays in obtaining ethical approvals for 
involving people and the cost of involvement in both time and money. One study team member 
reflected in the follow-up survey that they had “worried too much” about the time-burden of 
involving participants. The concern of “not putting further pressure” on participants was a theme in 
email communication between the study team when making decisions to limit contact with 


participants throughout the process of involving them. 


Survey data from the study team also suggests that adequate funding, a supportive team, involving 
participants in the very earliest stage of research planning and co-designing inclusive methods of 


involvement all contributed to the impacts reported. 
Section 3: Impacts on study design from stakeholder involvement 


Involving stakeholders in the co-design process impacted the study in seven specific impacts ways. 
By asking for participants’ views on aspects of the proposed study design, the study team gained 
insight into participant preferences and opinions. While there was diversity in views, the process 
allowed the study team to improve aspects of the study design. These impacts are summarised in 


Table 5.6. 


Table 5.6 ‘Summary of impacts on study design’ 


Impact on planned research Summary of impact 
1: Recruitment and sample Recruitment and consent for the MGRS will occur online 
collection wherever possible, and saliva samples (rather than blood) will 


sent by post to be used as biospecimens for DNA analysis. 
2: Participant communication A short video and ‘information pack’, which will explain the 
MGRS study, will be created to assist with recruiting family 


members. 
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3: Participant involvement in 
governance 


4: Data access 


5: Communication and ways of 


involving participants 


6: Provide feedback to participants 


about the research 


7: Create learning and development 


opportunities 
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Participants will be invited to be involved in overseeing 
governance, including funding decisions. 

Study participants should be involved in controlling data access 
decisions and policies. 

Participants will be included on study advisory groups, including 
for study recruitment and communication, data access and 
ethical oversight using multiple communication modes. 
Participants will be informed about the impact of the research, 
and how their involvement has affected the design and 
management of the study. 

Learning and development opportunities will be created for 


potential participants, researchers and other stakeholders. 


Impact 1: Recruitment and sample collection approach 


A discussion about participants’ adult children being ‘time-poor’ highlighted the importance of a 
study design which minimised the time burden on younger generations. Only a third of participants 
thought their children would be willing to do a blood test. This allowed the research team to make a 
more informed decision about ‘trade-offs’ between the data that can be collected via blood or 


saliva, versus the potential effect on recruitment. 
Impact 2: Participant communication 


Most participants reported willingness to be involved in recruiting family members to a new study, if 
given appropriate information and supporting documents. Relatively inexpensive information 
resources, such explanatory videos, could have a significant impact on the success of the 
recruitment and the study as a whole. One participant also advised that information produced for 


participants by researchers can be confusing, and that laypeople can help simplifying it. 
Impact 3: Participant involvement in governance 


Event participants unanimously agreed that they should be involved in all aspects of the research, 
whereas 11 of participants from the 20 analysed interviews thought that they should be involved in 
study design. One interview participant felt only researchers should be involved as they are “the 
qualified people” (P8). Five of the interview participants expressed the view that non-researchers 


are required in research as they provide an important alternative perspective. 


Participants had mixed views about commercial organisations funding research. Four event 
participants were against it, some were ambivalent, and the majority indicated no objections. One 
participant was concerned about the risks posed by involvement of commercial organisations with 
opaque “vested interests”, asking “what is in it for them?” (P2), referring to individuals and 
organisations with real or perceived conflicting, competing or commercial interests. Participants 
suggested that involving people in governance (including funding and ethical oversight) was a way of 


mitigating this risk. 


Study team members reflected that public funding would be preferable to commercial funding, as 
the responses at the event suggested that a commercially-funded study might negatively impact 
recruitment. As a result, the study team altered the proposed design to involve participants in 


governance, oversight and funding decisions. 
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Impact 4: Data access 


Participants shared the view that they would like different kinds of genetic information returned 
from the research (see Table 5.3), including personal medical, ancestry and pharmacogenomic 
results. Two-thirds of event participants wanted access to their own genomic data, and had mixed 
views about who else should have access. All event participants stated they were comfortable with 
their data being held by academics, and none were comfortable with data being held by a 
commercial company. However, one participant suggested not “ruling private companies out 


completely” from research (P9). 


General practitioners (GPs) were generally trusted to access and interpret genomic data, but 
participants felt GPs should not have access to data that they did not. All but two event participants 
agreed they should exclusively control access to their own data, with those disagreeing mentioning 


cognitive decline as a reason for a co-managed access model. 


Some participants had concerns about themselves or their biological relatives (especially offspring) 


finding out information they “might not want to know” (P6), or having it imposed on them. 
Impact 5: Communication and ways of involving participants 


Preferences for communication mode differed between interview participants and event 
participants. Interview participants preferred questionnaires via post (30%, 18/50), and face-to-face 
events (25%, 15/59), over online questionnaires (17%, 10/59) and online discussions (8%, 5/59). 
Event participants suggested face-to-face meetings were helpful but only when there was an 
“occasional need” (P3). Participant responses also suggested that limiting face-to-face events where 


possible (in favour of telecommunication) may mean involvement is more inclusive. 


Event participants felt certain tasks (such as reviewing information) could be done “more online” 
(P3). Participants spontaneously suggested using online, moderated forums and suggested that 
these should be hosted by trusted institutions (such as universities) rather than commercial 


organisations, as some ‘don’t trust’ social media companies [P17]. 
Impact 6: Provide feedback to participants about the research 


Event participants stated that keeping people informed about what has been learned from the study 
is a good way of keeping people engaged in the study and improving retention. For example, 
ASPREE-XT sends a regular newsletter to participants. Participants also stated that they would like to 


be informed about when their involvement has made a difference. 
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Impact 7: Create learning and development opportunities 


Participants often stated their willingness but also their uncertainty about how they could be 
involved. Learning and development opportunities were identified as an e of involvement by both a 


participant advisor. 
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Figure 5.2. Co-designing multi-generational genomics research 


Co-designing a multi-generational genomics study 


ASPREE has DNA from over 15K participants, Se ~ Je 
with 90% still attending annual visits 5 years on AÀ AÀ 


participants inviting them to a with 59 participants are 
get involved in co-designing study analysed, data informs event design 


18 participants attend ' 


co-designed event to Eo -data from the entire 18 month 
explore views about study FO process is co-analysed | @ 
preferences for involvement with stakeholders 


Changes to the proposed study design included: 
“> Recruitment done online, saliva 
‘ collected by post instead of blood samples 


Newsletter sent to 14 268 : Ti i i 
, AI £ elephone interviews 
2 


» Study team interviewed and 


Ds No data access for profit 


: : E 2e 
Involve participants online and in-person 
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Discussion 


The participatory action research method gave insights into participants’ preferences that 
measurably impacted on the proposed study design. The improvement of the interview design using 
the co-design process illustrates the value of a flexible and iterative approach to involvement in a 


study. 


Participants’ preference for being informed about both the study outcomes and the outcomes of 
their involvement is supported by other studies which suggest that communicating about the 
research regularly and sharing results may improve retention. 7123713” This is particularly relevant 


for those planning involvement in genomics research, which may span decades. 7” 


Such regular communication should also be combined with learning and development opportunities, 
which could also help facilitate participant involvement by ensuring that people understand the 


values which motivate the tasks they are being asked to be involved in. 27325 


Participant views about data storage and access aligns with a 2015 survey by Genetic Alliance 
Australia, which indicated people mostly trusted universities and research institutes to use personal 
genetic information for research, with commercial companies least trusted. 7”? Event participants’ 
unanimous preference for having access to their own data and a general trust for GPs to access and 


interpret genomic data also aligns with findings from other studies. °”* 37° 


Participants’ concerns about unintentional disclosure of data to biological relatives who ‘might not 
want to know’ certain information highlights the challenges of asking participants about information 
preferences, and recognises that this ethical decision extends beyond individuals [P6]. This issue is 
common to almost all ongoing and proposed genomics research studies and should be urgently 


addressed by contemporary individualist ethical frameworks. 37° 
Study strengths 


By asking participants their preferences, the study team gained useful insights to inform the design 
of the proposed study. Participants preference for being involved in decision making about funding 
sources, data management and ownership, and what information to share with participants will help 
ensure any future study design aligns with participants’ values, ensuring the design is culturally safe 


and culturally competent. ?”” Similarly, the participants preference for being involved in reviewing 
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participant information aligns with other research which suggests that involving potential 


participants in reviewing information can help improve recruitment. 33? 


The effective involvement of ‘stakeholders’ also includes involving all relevant staff and health 
professionals at all levels of an initiative, who may have important knowledge or perspectives that 


senior research staff do not. 


The ‘transformative learning’ during the process reported from both study participants and the 
study team was an important impact captured by the participatory action research (PAR) 
method.’”°!”” The process showed that it was valuable to create regular involvement opportunities 
for each stakeholder. Reporting this process in a standardised way using ‘Standardised Data on 
Initiatives’ (STARDIT) meant that impacts such as transformative learning could be reported and that 


this case study can be compared to similar studies in the future [49]. 
Study limitations 


Because the 76 participants who responded to the original advertisement were self-selected 
volunteers, our findings may not necessarily be accurate for the whole ASPREE-XT cohort. Our 
sample may reflect a sub-set of individuals who feel more strongly supportive of a proposed MGRS 


than the cohort average. However, the data from this process is still useful and valid.°” 


The number of interviews which were transcribed (20) and analysed was high for a case study. 378 As 
there is no agreement in case study literature on whether to code all or some of the data, our 
methodology balanced exhaustive examination of the data with the capacity bias imposed by time 
constraints. 237? Therefore, data transcribed and coded from the interviews does not include all the 


data collected in the interviews. 


While the mixture of written and verbal feedback at the event ensured a range of ways to give 
feedback, the voting process (which involved people raising their hands in front of everyone) may 
have given different results if it was an anonymous ballot.” Future research should explore and 
compare preferences for different methods of voting. 

While the process of involving people described here did not exclude people based on language, 2°° 
the original ASPREE study required a certain level of English language skills in order for people to 


participate in some of the cognitive assessments and public events. The process described here may 
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therefore have excluded people who cannot read English or do not feel confident speaking in 


English, such as people who speak English as a second language. 
Conclusions 


This case study provides evidence that including participants in the design of a research study 
positively impacted the study design. As many research studies are negatively impacted owing to 
poor recruitment and retention, such evidence is increasingly important for informing involvement 
in future studies. The process of involving ASPREE-XT participants in the design of a new MGRS was 
highly valued by stakeholders, and was positively impactful for both participants and the study team. 
The lead investigator stated “the feedback has been so valuable” and that it will be “built into the 
design” of future research. Learning from the case study suggests that adequate funding, a 
supportive team, involving participants in the very earliest stage of research planning and co- 


designing inclusive methods of involvement all contributed to the impacts reported. 


The process of involving people can be viewed as a learning experience for both the participants 
involved and study team members. The process changed participant and study team members’ 
views about the value of involvement. For example, one a study team member stated they ‘learned 
that participants have valid, interesting, and often novel ideas that researchers may not have 
considered’ and their ‘perceptions of involving participants in planning medical research have 
significantly changed’, which can be viewed as an impact of ‘transformative learning’. 17°72 Using a 
standardised reporting tool to describe the methods and impacts provides a way for learning from 
this case study to inform future research studies planning to involve people, including studies 


beyond the discipline of public health genomics. 


This is the end of the extract from the publication ‘Involving elderly research participants in the co- 


design of a future multi-generational cohort study’: https://doi.org/10.1186/s40900-021-00271-4 
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STARDIT data: Involving elderly research participants in the co- 


design of a future multi-generational cohort study 


This is an extract from a report which uses the Standardised Data on Initiatives Alpha version 


(STARDIT).*°2 A full version of this report can be found here?**: https://doi.org/10.1186/s40900-021- 


00271-4 


Involvement 


Who was involved 


1. 3 academic research investigators 
2. An ASPREE participant assessor 
3. An ASPREE-XT participant 


Specific tasks of this person or 


group (list as many as possible) 


Everyone listed above was involved in co-designing every stage of the 
process. This included refining wording of participant information, 
sharing views and advice about the process, proof-reading documents, 
providing feedback on questionnaires, analysing data, informing 
planning, presenting information to participants, interpreting data, and 


participating in email surveys. 


What was the outcome or 
output of the involvement of 
these people? What changed as 


a result of involving people? 


Improved participant information resources, improved wording that is 
culturally appropriate, improved question design for interviews, 
improved learning resources for participants, improved co-design 


process. 
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Three members of the study team were employed by Monash 


University during this process. 


N/A 


3. Learning from this process will be presented at conferences, 
shared on social media and through other channels (such as 
podcasts). 

How is the data FAIR?®? Data will be shared in the public domain under a Creative Commons. 


Impacts and outcomes 


What new knowledge has been 
generated? (if appropriate, 
include effect size, relevant 


statistics and level or evidence) 


Involving participants in co-designing a proposed study resulted 
in changes to the design of the proposed study 

The process of involving people can be viewed as a learning 
experience for both the participants involved and study team 
members. The process changed participant and study team 
members’ views about the value of involvement, which can be 


viewed as an impact of ‘transformative learning’. 


Outcomes - Describe how the 
learning or knowledge 
generated from this initiative 


has or will be used 


Knowledge from this process will inform the design of a future 
multi-generational study 
Learning from this process can inform future involvement 


activities 


Ethics approval and consent to participate 


The Ethics Committee of the Alfred Hospital (Melbourne, Australia) granted approval for this project, 
which was considered Low Risk Review and approved on 16/04/2018. 


Project No: 180/18. 


Project Title: ASPREE Genomics Engagement: Planning of Future Multi-Generational and Family 


Studies. 


Principal Researcher: Dr. Paul Lacaze. 


Participants gave informed consent to participate, with further consent being granted by 
participants for sharing of recordings from the process. 
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Chapter 6 — Involving people affected 
by a rare genetic disorder in shaping 


future research 


This chapter is adapted from the published, peer-reviewed 2021 article titled ‘Involving people 
affected by a rare condition in shaping future genomic research’.’ Please note: 
e Figure numbers have been changed to align with this thesis, and some formatting has been 
adapted, including integrating references into this thesis. 
e This paper uses the term ‘enablers’, to avoid confusion with those facilitating discussions, 
where previous chapter used the term ‘facilitators’. 
e The published version can be found here, along with the additional files: 


https://doi.org/10.1186/s40900-021-00256-3 
Authors 


Jack S Nunn Kylie Gwynne, Sarah Gray, Paul Lacaze 
Plain English Summary 


There is evidence that involving potential participants and the public in co-designing research can 
improve the quality of the study design, recruitment and acceptability of the research, but 
appropriate methodologies for doing this are not always clear. This mixed methods study examined 


ways of involving people affected by a rare disease in shaping genomic research. 


In this article we describe how people were involved in the co-design process, and ways to plan, 
report and evaluate involvement in research, including impacts. We demonstrate using a novel way 
of sharing data in a standardised way to plan, report and evaluate how participant involvement 
activities positively impacted the study design (Standardised Data on Initiatives - Alpha Version 0.1 - 
STARDIT). STARDIT is an open access data-sharing platform being developed to standardise the way 


that information about initiatives is reported. 
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We conducted pre and post surveys and facilitated online discussions. There were six outcomes from 
the process, which included participants reporting an improved understanding about how to get 
involved in research and that learning resources were useful. Participants reported changed views 
about involvement, with most participants ‘widening’ their perception of who should be involved in 
research to include more people. Participants enjoyed online discussions and asked to stay involved 
in the research process. The partner organisation reported that similar online discussions will be 
used in future research prioritisation processes. Standardised reporting of this study will help inform 


the future involvement of participants and the public in the design and conduct of genomic research. 


Abstract 


Background 


There is evidence that involving potential participants and the public in co-designing research can 
improve the quality of the study design, recruitment and acceptability of the research, but 
appropriate methodologies for doing this are not always clear. In this study we co-designed a way of 
involving people affected by a rare genomic disease in shaping future genomic research about the 
condition. The aim was to report the process, experiences and outcomes of involving people in 
genomic research in a standardised way, in order to inform future methods of involvement in 


research co-production. 
Method 


Participants were recruited from an online community hosted by an Australian-based rare disease 
charity and were over the age 18 years. Once people gave consent, we shared learning resources 
with participants and invited them to complete an online survey before joining a two-week 
facilitated online discussion, followed by a second online survey. We used the novel tool 
‘Standardised Data on Initiatives - Alpha Version 0.1’ (STARDIT) to map preferences, plan 
involvement and report any outcomes from the process, with quantitative data analysed 


descriptively and qualitative data thematically analysed. 
Results 


Of the 26 people who gave consent and completed the initial survey, 15 participated in the online 
discussion and 12 completed the follow-up survey. STARDIT was used to report six outcomes from 
the process, including 60% of participants’ responses showing a change towards ‘widening’ their 


view of who should be involved in research to include more people. Outcomes also included an 
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improved understanding of research and how to be involved. Participants enjoyed online 
discussions, found learning resources useful and asked to stay involved in the research process. The 
partner organisation reported that a similar online discussion will be used in future research 


prioritisation processes. 
Conclusion 


Involving people in co-designing the process improved the study design, ensuring it met the needs of 
participants. Whilst the study includes participants from only one disease group, using STARDIT 
allowed us to map people’s preferences and report the methods and outcomes from involving 
people, providing a way for learning from this case study to inform future research studies beyond 


the discipline of public health genomics. 
Introduction 


Genomic research may lead to better understanding and management of the Eosinophilic 
Gastrointestinal Disorders (EGID), including Eosinophilic Oesophagitis (EoE). EGIDs are long-term 
(chronic) inflammatory conditions that affect the lining of the throat, stomach and gut (epithelium). 
EoE is the most common type of EGID and is most likely caused by exposure to food antigens.**" EoE 
affects people of all ages, gender and ancestral backgrounds.**? Genomic research may lead to 
better understanding and management of the disease.*®? EoE is a rare disease and involving 
people affected by EoE in shaping future research could help ensure that the research is relevant, 
well-designed and aligned with patient priorities.**°” However, currently there is no standardized 


way of planning and reporting how people are involved in shaping future genomics research.”’ 


This illustrative case study documents a participatory action research process with the charity ausEE, 
to co-design a way of partnering with people affected by EoE (including their carers), to help shape 
future research. The study aimed to examine the processes, experiences, barriers and enablers of 
participant involvement in genomic research about one condition EoE to inform methods of genomic 
research co-production. In this article we aim to outline how people were involved in the co-design 
process, and how Standardised Data on Initiatives (STARDIT) can be used to plan, report and 


evaluate involvement in research, including impacts.384 


We created a prototype online discussion forum, guided by the principles of participatory action 


research (PAR) from the International Collaboration for Participatory Health Research, and guidance 
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on co-design. 16366 This allowed us to explore and report the practicalities, enablers and barriers of 


using an online discussion as a way of involving people in genomic research. 


Involving the public, patients, research participants and other stakeholders in actively contributing to 
the research process can lead to a range of positive outcomes. These can include improving the 


t?°3 quality and relevance of research. 7”? Involvement is when research is carried out 


recruitmen 
‘with’ people rather than ‘on’ them. 7° In human genomics research, the need to involve the public 
and other stakeholders is a crucial aspect of responsible research practice, which can help ensure 
outcomes of importance to all stakeholders are included in decision making processes.7°?”3338 The 
term ‘stakeholder’ means anyone who has a ‘stake’ in the research, in particular those with 
important knowledge, experiences, expertise or views that should be taken into account. 2°?° This 
can include people affected by Eosinophilic Oesophagitis (EoE), the study team, and the wider 
public, although in this process we did not involve the wider public. At the earliest stage of the 
research cycle (the conceptual stage), people affected by EoE were involved in the co-design of this 


study. 
Methods 


In order to co-design the study from an early stage, a representative from ausEE was contacted, 
inviting the organisation to partner with the research team, with a representative from ausEE invited 
to be part of the study team and another invited to give feedback on the proposed study design. The 
representatives were involved in a number of tasks including reviewing and improving the written 
information, online survey questions, and the facilitation plan for the online discussion. 

In order to facilitate comparison with other studies, we used the novel system ‘Standardised Data on 
Initiatives - Alpha Version 0.1’ (STARDIT) to map preferences, plan involvement and report any 
outcomes from the process.’ This included reporting co-design positively impacted the study. 
STARDIT is an open access data-sharing platform being developed to standardise the way that 
information about initiatives is reported across diverse fields and disciplines, including information 
about which tasks were done by who. Quantitative data was analysed descriptively and STARDIT was 
also used in parallel with a thematic analysis, which organised data into pre-defined ‘super- 


categories’ which allow consistent comparison with other data using STARDIT. 


Learning resources were both co-created and selected by the investigator team, working in 
partnership with the Australian Genomics Health Alliance and co-refining the selection with 


potential participants, including working in partnership with potential participants and the Australian 
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Genomics Health Alliance, using the outcomes of a landscape analysis of educational materials as a 
starting point for selecting resources.?*° Final resources were checked by a medical professional 


specialising in EoE. 
Study Design 


A participatory action research (PAR) paradigm was chosen to guide the process with co-design and 
reporting informed by guidance from a number of sources.’?*”1?3 Participatory action research is an 
umbrella term which describes a number of related approaches, including forms of action research 
which embrace a participatory philosophy and include ‘co-design’ and ‘co-production’ of research. 
84 It is a process whereby researchers, the public and other relevant stakeholders “work together, 
sharing power and responsibility from the start to the end of the project”,®° including knowledge 


generation and translation.®° Elements of this study were co-designed in parallel with another similar 


study, therefore some aspects were inflexible and thus ‘co-refined’ rather than ‘co-designed’. 


We used case study methodology to record and describe our experience involving participants in an 
online discussion about genomics research. The selection of the population for this case study was 
informed by a number of factors which were appraised by the study team, including ethical, 
pragmatic and population considerations.’’”*7° One of the investigators (PL) had a professional 
relationship with the charity ausEE, which was used as a starting point by the study team to explore 


the appropriateness of the case study. 


The case study is presented as an instrumental case study, where the purpose is to understand the 
particular case and can attempt to provide data that could produce useful generalisations by using 
inferences from the data." We collected and analysed both qualitative and quantitative data 


during the involvement activities, informed by a number of frameworks and standards.7°?3™ 


In addition, we analysed other data from participant survey responses, online discussions, meeting 
notes, emails, surveys of the study team and reflexive diary entries of one member of study team 
(JN). Coding and thematic analysis of qualitative data was carried out by two authors independently 
(JN, KG) and checked by another author (PL), following best practices for enhancing validity in 
qualitative methods.’”® Two authors of this paper also shared personal comments in the online 
discussion (KG, SG), which have been anonymised using participant numbers. In order to aid analysis 
and comparison with other case studies, we used a novel way of sharing data in a standardised way 


(Standardised Data on Initiatives - Alpha Version 0.1) to map preferences for involvement, plan 
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involvement, report and evaluate how people were involved in the participatory action research 


process, and how this positively impacted the study design.*% 


STARDIT is an open access data-sharing platform being developed to standardise the way that 
information about initiatives is reported across diverse fields and disciplines, including information 
about which tasks were done by who. It also offers a way to add updates throughout the lifetime of 
an initiative, from planning to evaluation and reporting any impacts. Authors from this paper were 
involved in co-creating STARDIT, and learning from this process informed the development of the 


reporting tool.?° 
Participants and recruitment 


This study recruited participants from an existing online community managed by the Australian- 
based charity ausEE. A link to the informed consent form and learning resources was shared by 
ausEE ona closed Facebook group. We recruited people if they were either a parent, a carer, a 
partner, family member or loved one of someone with EGID who is under 18; or someone who was 
over 18 with EGID. If people gave consent, they were invited to complete an online pre-discussion 


survey and sent instructions for joining the online discussion. 
Stages of research 


There were four stages of the research process: 1. Co-design; 2. Recruitment and surveys; 3. Online 
discussions and 4. Evaluation and reflection. The multiple stages of the co-designed research are 


summarised in Figure 6.1. 
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Figure 6.1: Stages of research 


Stage 2: Recruitment 


September October October October December 


AusEE Study team Study team co- Invitation to Survey, online 14 day online discussion with 15 _ Post-discussion Analysis 
representatives established with design study participate discussion details and participants, with simultaneous Survey sent complete 
invited to co- AusEE shared on learning resources Facilitator support group 
design study representatives social media sent 


At each stage of the process, stakeholders were involved in the following tasks: 


P is CAN ABE OÈ a A P Ue 


Checking Co-creating 
wording learning resources 


Initial contact was made with ausEE in July 2018 and two investigators from ausEE were invited to 


join the study team (SG, KG). After ethics approval was obtained, an invitation to participate in the 
study was shared in the ausEE Facebook group and member newsletter in September 2018. This 
group has members who live in both Australia and New Zealand. Before being asked to give consent, 
the study was summarised in plain English and two learning resources were shared with participants 


in order to provide context to the study (see Additional File 1 ‘Data and analysis’). 


The invitation to participate contained a link to the participant information document and the 
informed consent form. If people gave consent to participate, they were invited to complete an 
online pre-discussion survey, which had demographic data categories informed by other similar 
studies to allow comparison.” 

Participants who gave consent were then contacted by email, with information about joining the 
discussion shared alongside relevant learning resources. Participants were also sent a follow-up 
survey after participating in the online discussion. Questions relating to ‘Who should be involved in 
research’ were the same as in the pre-discussion survey to allow comparison. Participants could 
choose from the categories outlined in Figure 6.2, with a change in direction towards more people 


being involved labelled as ‘widening’, the inverse as ‘narrowing’. 


Figure 6.2: Who should be involved in research? 


Anyone who might Only people who 
be indirectly affected are directly affected 
by the research by research 


Online discussion methodology 


We conducted an online discussion to maximise flexibility about when participants could contribute 
(for example, people in different time-zones or those with different caring responsibilities). A 
significant advantage of online discussion platforms is that they are accessible to a greater number 
of people. This creates a more equitable platform for people to participate, compared to a 
synchronous (simultaneous ‘real-time’) discussion. 

We offered anonymity in online discussions to avoid people inadvertently disclosing sensitive or 
personal information, which may have serious implications, for example, impacting on eligibility for 
future health insurance. Participants could choose to use their real name and email address, or 


remain anonymous by using pseudonyms or temporary email accounts. 


Before participants joined the online discussion, seven learning resources were shared with them. 
This included a short 60 second online video about the study context and purpose??”, a one-page 
infographic summary of a scoping review about genomics research*®, and a short two-page 
summary of genomics and contemporary research relating to EoE was co-created with ausEE, the 
study team and experts in genomics (see Additional File 1 ‘Data and analysis’).?°° 


An open-source software web application ‘Loomio’?° 


was installed on virtual machines hosted by an 
Australian Government initiative called ‘Nectar Cloud’ .3?t Participants could securely log-in from 
anywhere in world and participants’ data was stored securely on servers physically located inside 


Australia. 


Two people facilitated and moderated the discussion for 14 days (JN, KG), one of whom was a parent 
of a person affected by EoE (KG). Participants were invited to co-create their own boundaries for the 
group discussion at the start by reviewing provided statements and suggesting amendments. The 
facilitators judged when to introduce new topics (depending on the engagement with each topic) 


with the recommended schedule below in Table 6.1 used as a template. 
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Table 6.1: Online discussion overview 


What do you understand by the word ‘research’? Day 1 
What do you understand by the phrase ‘genomic research’ ? Day 1 
Why do we do research? Day 2 
Which aspects of any future genomic research should be influenced by the following groups of Day 4 
people? 

What methods do you think could be used to involve those people in future genomics research? Day 6 
Do you have any ideas, thoughts or reflections that have not been shared yet? Day 7 
Discussion closed Day 14 
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Results 


Figure 6.3: Process summary 


this study was co-designed with people affected 
by rare Eosinophilic Gastrointestinal Disorders 
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Stage 4: Evaluation 
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Stage 1: Co-design 


The input of the representatives during the planning and co-design stage had clear positive impacts, 
particularly in improving educational resources and ensuring the online discussion was advertised 
using wording appropriate to the existing online community. For example, representatives from 
ausEE helped change the study design to include explicit opportunities for participants to learn more 
about genomics and EoE, avoiding participation being perceived as having a one-way benefit. During 
the co-design process it was also decided to exclude people who were under 18 and people who 
stated they were representing someone who was over 18, as people who were 18 and over had the 


choice to represent themselves. 
Stage 2: Recruitment and surveys 


In total 26 participants completed the pre-discussion survey, 12 completed the follow-up survey. 
These responses are summarised in Table 6.2. All but one of the participants were female, with most 
reporting they were parents of a person with an EGID. Self-reported educational attainment was 
mostly ‘degree (bachelors), diploma or post-graduate’, with one participant reporting they had 
professional experience in genomics. Most participants were between 30-45 years old and 


participants all lived in Australia, except for one who lived in New Zealand. 


A total of 41 responses were given by 12 participants who shared their identity at both the baseline 
and follow-up, including ten questions about which aspects of genomic research should be 
influenced by different stakeholders which identical at each stage. Of the 41 responses to the ten 
questions, 60% showed a change towards ‘widening’ involvement (N=25/41), 36% of responses 
stayed the same (N=15/41) and 7% ‘narrowed’ (N=3/41). Recurring themes were identified by the 
study team, and six specific outcomes were reported as a result of involving participants in this 


process. 
Stage 3: Online discussions 


A total of 15 people participated in the online discussion. The President of the charity ausEE also 
identified herself by name in the discussion (SG). All but two participants chose to use their real 
name in the online discussion, which they had provided when registering and giving consent. A visual 


representation of the online discussion is illustrated in Figure 6.4. 
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Figure 6.4: Online discussion visualisation 


This text has been edited from the original for brevity. 


Stage 4: Evaluation and reflection 


Participant feedback about the online discussion was sought via survey and was positive, with one 
participant describing the process as ‘very good’ [P25]. Participants reported that they enjoyed being 
involved and that participating in the discussion changed their views about involvement. The visual 
summary of the scoping review about involvement in genomics that was shared at the start of the 
discussion was described as a ‘great summary document’ [P21]. One facilitator noted the group 
were ‘well informed’ about research [P16], with the discussion indicating a high level of general 
understanding about research, genomics and the associated ethical, legal and social implications. For 
example, one participant stated ‘I wonder about the limitations of genomic sequencing but believe 
knowing the result is an adult human right’ while adding they were unsure ‘how much’ they would 
want to know themselves [P25]. Another participant mentioned the ‘moral minefield’ of pre- 


implantation genomic screening, noting that “not all that is ‘marked’ will come to be” [sic] [P3]. 
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Table 6.2: Summary of pre and post survey responses 


What made you decide to 
respond to our invitation to 
participate in this project? 


22 responses (pre-discussion) 


What do you hope to get out of 
participating in this discussion? 
Do you have any specific 
expectations? 


20 responses (pre-discussion) 


Do you have any ideas about how 
the different people could 
influence future research? 


19 responses (pre-discussion) 


Is there anything in particular you 
liked or thought was helpful 
about how the discussion was 
conducted? 


8 responses (post-discussion) 


Is there anything you didn’t like, 


thought was unhelpful or could 
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Fourteen participants stated they decided to participate as they 
wanted to help improve knowledge of the disease and help find a 
cure. Two people were specifically interested in genomic 
research [P6, P9], and one person reported they were 


‘researching themselves’ and their sons’ genomic variations [P5]. 


Four participants wanted to ‘learn more’ [P16]. Five participants 
stated they wanted better outcomes for patient care and 
treatment protocols, with three participants stating they wanted 
to be actively involved in helping research to improve outcomes. 
Two participants wanted to hear the perspectives of others. Five 
participants stated they had no expectations. 

One participant stated that people with a rare disease and their 
families are ‘likely to have different priorities from scientists’ 
[P16]. Another suggested that ‘sharing patient experiences, 
priorities of research areas’ and involving people in co-defining 
‘ultimate patient outcomes’ were ways people could influence 
research [P21]. 

One participant said they ‘enjoyed the interaction, helpful links 
with information about genomics and the topic threads’ [P21]. 
Another added they liked ‘being able to read others thought 
processes on each topic’ [P3]. Others said ‘the responses to mine 
were timely and provoked further questions that made me think 
in new directions’ [P25]. Two participants said that ‘having 
different topic questions/threads was helpful’, as was the 
‘information provided to start’ of some threads [P29]. One 
participant added that ‘it was great because even though busy 
with my son’ they could travel and ‘still catch up and learn things 
and do my input too’ [P30]. 

Only two participants provided answer for this question other 
than ‘no’. One participant stated ‘I didn't like the platform it was 


conducted on’ as it was not ‘user-friendly’ [P15]. Another added 


have been improved about how 
the discussion was conducted? 


4 responses (post-discussion) 


Did you have any expectations 
from participating in this research 
that were met or not met? 


8 responses (post-discussion) 
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that sometimes facilitators ‘added another question too quickly 
before a number of people had a chance to answer the first one’ 


which risked leaving ‘some people behind’ [P3]. 


One participant stated that they wanted to ‘learn more about the 
difference of the gene and genomics and that was met’ [P30], 
and three others stated their expectations had been met, with 


none reporting they had not. 


Outcomes from the process 
There were six outcomes reported from this process, which are summarised in Table 6.3. 
Table 6.3: Summary of outcomes from the process 


Outcome Summary 

1. Learning resources useful Participants stated that the learning resources were useful and 
helped improve their understanding of genomics, research and 
the associated ethical, legal and social implications. 

2. Changed views Participants reported that the process of being part of an 
online discussion gave them an opportunity to ‘learn a lot’ 
about the views and experiences of others and reflect [P25]. 
Participants reported that this ‘challenged’ them into 
‘rethinking’ or changing or their own views on certain issues 
[P25]. 

3. Enjoyed online discussions Four participants said they enjoyed being part of the process 
and preferred it when compared with other methods such as 
face to face discussion or interviews. 

4. Online discussions to be used The President of ausEE stated that the discussion had ‘given 

in future research prioritisation her an idea’ and that she would combine face to face and 
online discussions into a process to involve people in deciding 


how the medical research fund is spent. 


5. Participants asked to stay Participants requested to stay involved in the research process, 
involved in the research including in analysing data and being co-authors on the paper. 
6. Improved understanding of Participants reported improved understanding of genomics 


how to get involved in research research, including how they could be involved. 
Participant views about involvement genomic research 


While the focus of the online discussion was involvement in genomics research, the themes of the 
discussion reflected the interconnected nature of the subject, including ethical, legal and social 
implications of genomics research. Issues discussed included research prioritisation and funding; 
data sharing; health technology assessment, and health and life insurance. Fourteen participants 
shared the view that they wanted to be involved in improving knowledge of EGID and helping find a 
cure ‘so other parents don't go through what we went through’ [P25]. Similarly, participants wanted 


to help genomic research in the area improve the lives of people affected by an EGID. 
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Ten participants identified specific enablers of involvement in genomic research, including ‘equity’. 
Open-source data and information was identified as an enabler of involvement [P3]. Emotional 
connection to an issue was identified as being either an enabler or a barrier to involvement, with 
one participant stating ‘there needs to be a balance’ [P21]. Other enablers included participants 
being ‘able to contribute without putting their personal situation at risk’ with data ‘well protected so 


as not to be used against an individual or group of people’ [P3]. 


The theme of the ‘intention’ and ‘purpose’ of research emerged, with a discussion on vested 
interests. One participant identified that people affected by EGID ‘certainly do’ have a vested 
interest [P3]. Participants were concerned that research priorities were driven by money. Four 
participants mentioned ‘insurance’ as an area of concern which requires public involvement and 
scrutiny with regards to data sharing, with one stating it is a ‘valid community fear’ that research 
data will be used ‘for’ insurance companies or other for-profit organisations [P3]. One participant 
stated that ‘progressive watering down of privacy protections’ meant they were concerned their 
children would be ‘refused insurance because of a decision | have made’[P16], such as participating 


in research. 
Stakeholder views about the online discussion 


Participants identified specific things about the online discussion which they felt enabled 
participation or were barriers to participating. Participants ‘enjoyed the interaction’ of the online 
discussion and were ‘supportive, positive and open’ [P21] [P3]. Four participants found the links to 
learning resources helpful. Participants felt online discussions are ‘good to help’ people get involved, 
in particular those who are unable to travel or live in remote areas [P20]. The research method used 
in this online discussion was described as ‘very good’ as the time frame allowed for ‘life 
responsibilities and also gave time to process new information and think about others’ comments’ 
[P25]. However, one participant and one facilitator felt the discussion was ‘too quick’ [P3]. Other 
participants reported the length and pace was good, highlighting that co-design is an important 
element in designing similar future discussions. One participant stated that ‘this style of discussion 
here has been interesting and by learning from others | have found some of my initial thoughts have 
changed’ [P25]. Other participants liked the idea of a multi-stage way of involving people using a 
combination of face to face and online discussion [P25], which aligns with other models used in 


priority setting.1*?° 
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One participant mentioned that ‘one of the things that | really love about this research is that the 
research topic is on involving people in research and this research itself does exactly that - involves 
people in the research’[P21]. The ‘open discussion’ at the end where participants could raise any 
subject or question was also highlighted as a ‘helpful way forward in research’ as it allows 
participants to identify and focus on important areas, and researchers an opportunity to ask 
clarifying questions, additional questions and re-focus larger research questions. Other participants 
noted they might be influenced by the answers of others and that as this is part of the research 
process, it would need to be a consideration in data collection methodology [P21]. For example, 
people might share different information in an anonymous survey compared to an online discussion, 
owing to concerns about privacy or because another participant has said something which has 


prompted a memory they might otherwise have forgotten. 
Discussion 


Involving participants in co-designing the research process resulted in changes to the study design, 
including improving language used in recruitment and learning resources. Involving people in online 
discussions about involvement in research changes people’s views about who should be involved in 
research, including participants ‘widening’ their views about who should be involved in research to 
include more people. 

Most participants did not seek anonymity during online discussions, and this may reflect that 
participants were recruited from a social media group where their identities were already known 
and a sense of community already developed. It may also suggest that participants wished to be 


known to other participants in order to give and receive support inside and outside of the discussion. 


However, optional anonymity in online discussions can also be viewed as an advantage by some 
participants, by removing elements of identity which may be associated with power disparity, such 
as appearance, age or gender or future discrimination by health insurance providerr?”?. Anonymity 
for some may therefore lead to a disinhibition effect, with participants feeling more able to express 


views and perspectives that they may not feel confident sharing in other contexts. 
Learning from the outcomes 


The participants were highly engaged in the process, possibly as all self-reported a personal 
motivation for improving research in this area. The participants enjoyed being part of the process 
and none reported difficulties with the platform, which may be a result of familiarity with other 


online platforms, which may be correlated with self-reported higher degree level education. One 
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participant wrote ‘Thanks for this platform, process, everything really I've enjoyed being part of it’ 


[P21]. 


However, one participant said that while diagrams and visual aids were helpful, they wanted ‘more 
diagrams when reading through concepts or ideas’ [P3]. This suggests that online text-based 
discussions might not suit those with a preference for visual communication and highlights the 
importance of co-designing online discussions with potential participants who may have diverse 


learning or access needs. 


The learning from ‘Outcome 2: Changed views’ aligns with other studies, showing that the act of 
being involved in a discussion or in research can have outcomes which are a form of ‘transformative 
learning’ for both participants and study team members.*®” At the core of the participatory action 
research method is ‘critical reflexivity’, a process which asks people involved to reflect on the causes 
of problems, any solutions and the actions that people can take to improve the current situation.17° 
It is a form of collective self-reflective enquiry undertaken by people in order to understand their 
situation from multiple perspectives.1?? In this sense changed views of both researchers and 


participants involved can be viewed as an impact of ‘transformative learning’, with the reflexive re- 


examination of views and perspectives part of the participatory action research paradigm. 
Strengths 


While this study was planned and completed before the COVID-19 pandemic, the methods of 
involving people online described here now have an unexpected relevance to many disciplines, as 
research projects around the world seek to involve people online in novel ways, and evaluate such 


methods in a standard way. 
Using STARDIT for standardised preference mapping, planning and reporting of involvement meant 
outcomes from the process could be mapped more effectively??°**2, including outcomes from this 


process beyond the date of this publication. 


Measuring outcomes such as ‘transformative learning’ can be challenging. This process overcame 


such challenges by using baseline and follow-up surveys. 


It is significant that some participants preferred the online discussion method over face to face 


discussion or interviews, as this highlights the importance of using STARDIT-PM to map the potential 
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participant’s preferences when co-designing involvement, as this helped ensure research design 
meets the needs of those participating and those involved. Authors from this paper were involved in 


co-designing STARDIT, and learning from this process informed STARDIT Beta Version 0.2. 


In order to share power effectively and ensure this article reflected everyone’s experience of the 
process, the two people affected by EGID who were participant representatives were invited to be 
co-authors of this article. As participant representatives, they were involved from the very earliest 
stages of this research project. In addition there were opportunities for other participants to give 
feedback on the manuscript (including checking data analysis). In order to demonstrate the value of 
participant representatives’ contributions, the study team ensured they were invited to propose 
their own flexible time frames to contribute effectively to the study. This flexibility, and flexible 
deadlines from the article publisher in response to COVID-19, helped ensure they could balance 


providing feedback on data analysis and article versions with any caring responsibilities. 
Limitations 


The study design allowed people to be anonymous at each stage, so there was limited data from 
people who shared identifying information at each stage, meaning that some anonymous 
participants might be counted twice. However, as this study is a relatively small sample, statistical 
extrapolation from this case study is limited. This study also recruited from an existing online 
community which used a social media group, so data may reflect the views of people who have more 


experience using online platforms than the general public and are thus more competent. 


Some participants demonstrated passive behaviour in online discussions. For example, some 
participants logged-in multiple times, read comments but did not contribute comments. While the 
follow-up survey attempted to capture views of these participants, it was not completed by 
everyone, so it is hard to assess why certain people did not contribute more. However, one 
participant reported learning while reading and not contributing, and therefore there may be under- 


reporting of some learning outcomes from participants who were more passive .°% 


This study was designed in parallel with another similar study to allow comparison of data sets’, so 


some aspects were inflexible (such as the choice of open-source discussion platform) and could only 


be ‘co-refined’ rather than ‘co-designed’. 
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The decision to host the discussion on a separate platform (Loomio) from the one hosted by ausEE 
(on Facebook) was made for ethical reasons, as the study team could ensure control and ownership 
of the data. This may have been a barrier to some who decided not to participate, perhaps as the 
new platform was unfamiliar. However, data on this was not collected. While the gender balance of 
the participants was reflective of the wider online community from which participants were 
recruited at the time, data on educational background was not available so it is unclear if the 
educational background of participants was statistically representative. Future studies should seek 
to explore ways of appraising who might be excluded from research using more standardised 


processes.’ 
Conclusions 


This study provides valuable insights to the involvement of participants in research co-design and its 
associated benefits for researchers and participants. The co-design process improved the design of 
the study and ensured it met the needs of participants. Through sharing and discussing views in an 
asynchronous online discussion, participants reported their views changed. This aligns with other 
studies and demonstrates that transformational learning occurs through the process of involving 
people. After participating in the online discussion, people’s views about who should be involved 
‘widened’ to include more people. Participants wanted to be involved in shaping future genomic 
research in order to improve it, especially being involved in ethical oversight and scrutiny to improve 


safeguards regarding data use and privacy. 


Whilst the study includes participants from only one disease group, using a standardised reporting 
tool allowed us to map people’s preferences and report the methods and outcomes from involving 
people. Such reporting also provided a way to report the benefits to both participants and 
researchers, providing a way for learning from this case study to inform future research studies 


beyond the discipline of public health genomics. 


Contributions 


JN wrote the manuscript and analysed the data, facilitated online discussions, designed and carried 
out the process, evaluation of the process and the reporting (including STARDIT and GRIPP2 reports). 
KG gave feedback on the process design, facilitated online discussions, advised on methodology and 
paradigms, edited the manuscript, analysed data and checked data analysis edited the manuscript. 


SG was a participant advisor during the process and gave feedback on the process design, 
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participated in the process and gave feedback on the manuscript. PL gave feedback on the process 
design, checked data analysis and edited the manuscript. All authors helped co-design the process, 
read and approved the final manuscript. All participants who opted to stay involved were invited to 


give feedback on this manuscript. 


This is the end of the published peer-reviewed article ‘Involving people affected by a rare condition in 


shaping future genomic research’.’ 


STARDIT MICRO Report: Involving people affected by 


a rare genetic disorder in shaping future research 


This is an extract of data from a report which uses the Standardised Data on Initiatives Alpha version 
(STARDIT).**? A full version of this report can be found here”: https://doi.org/10.1186/s40900-021- 
00256-3 The WikiData version can be found here: https://www.wikidata.org/wiki/Q100403236 


Involvement 


Who was | Group 1: Academic research investigators (Jack Nunn and Paul Lacaze) 

involved Group 2: People affected by the rare disease representing the charity ausEE with experience of 
academic research (Kylie Gwynne) 

Group 3: People affected by the rare disease representing the charity ausEE (Sarah Gray) 
Group 4: People affected by the rare disease who are members of the online community and 


participated in the study 


Specific Group 1 and 2: Involved in co-designing every stage of the process, analysing data and member 
tasks of checking during the thematic analysis 

this Group 3: Involved in co-designing the recruitment and giving feedback on the proposed study 
person or | design and as an author of the paper. 


group Group 4: Invited to give feedback on the paper 


Outcome | Improved participant information resources, improved wording that was culturally appropriate 


S (using terminology preferred by the group to describe themselves), improved online discussion, 
(from improved learning resources for participants, improved co-design process. 

involvem 

ent) 


Two investigators are affected by the rare disease, which is 


why they were invited to be part of the study 
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N/A 


Data 

Who is the data from this It will be published open access in peer reviewed journals with identifying 
intervention shared with? information removed in order to anonymise it as much as possible. 

How is it stored and hosted? It will be shared on a public domain repository. 

Who is Group 1-3: The study team described above 

analysing the Group 4: participants were invited to review the analysis and give feedback to ensure they felt 
data? it reflected their experience of the process 

How is 1. It will be published in an open access journal 

information 2. It will be shared with participants of the research and also other members of the 
about this sibling group who have joined it since the study commenced 

data 3. Learning from this process will be presented at conferences, shared on social media 
disseminated? and through other channels (such as podcasts). 

How is the data FAIR°°? Shared in the public domain licensed under a Creative Commons. 


1. Involving participants in co-designing the research process resulted in a number of changes to 
the study design, including improving language used in recruitment and learning resources 

2. The process of involving people can be viewed as a learning experience for both the 
participants involved and study team members. The process changed participants’ views 
about who should be involved, which can be viewed as an impact of ‘transformative 


learning’. 


3. Learning from this process informed subsequent discussions in the charity ausEE about 


involvement in research, including proposed improved ways of involving people 


Ethics approval and consent to participate 


The La Trobe University Human Ethics Committee approved this study. 
Project number: HEC18242 


Project Title: Genomics Research and Involving People: ausEE 
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Chapter 7 — Co-designing genomics 
research with a large group of donor- 


conceived siblings 


This chapter is adapted from the published, peer-reviewed 2021 article titled ‘Co-designing genomics 
research with a large group of donor-conceived siblings’,*. Please note: 
e Figure numbers have been changed to align with this thesis, and some formatting has been 
adapted, including integrating references into this thesis. 
e This paper uses the term ‘enablers’, to avoid confusion with those facilitating discussions, 


where previous chapters used the term ‘facilitators’. 


https://doi.org/10.1186/s40900-021-00325-7 
Authors 


Jack Nunn, Marilyn Crawshaw, Paul Lacaze 
Plain English Summary 


Human genomics research is growing rapidly. There is evidence that involving potential participants 
and the public in co-designing research can improve the quality, recruitment and acceptability of the 
research. However, more evidence about effective methods for involving people is required, 
especially those in sub-populations which are inherently high interest to researchers and thus with a 


higher risk of being exploited by medical researchers. 


In this study, we worked with a large group of donor-conceived siblings who were conceived from 
the same sperm donor. We sought their views regarding participation in possible future research. 
We co-designed a way of involving them in discussion about their own “terms of engagement” with 
research. Online discussions gave group members an opportunity to share their views, and take 
initial steps towards developing their own research governance model. We used the ‘Standardised 
Data on Initiatives (STARDIT)’ Alpha Version to report involvement, allowing findings to be compared 


with other studies. 
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Group members who participated reported enjoying the experience and identified some advantages 
to online discussions over other methods, including time to reflect on answers and learn collectively. 


Most participants’ preferences about who should be involved in research design ‘widened’. 


Participants’ learning from the process also informed subsequent discussions in the sibling group 
about participation in research, including about how to make informed decisions about participating. 
Involving people in this way has the potential for a ‘transformative learning’ impact, where people’s 
perspectives and attitudes change as a result of being involved. This is particularly important for 
people in populations at greater risk of being exploited including under-represented sub-populations 


which are of high interest to genomic researchers. 
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Abstract 


Background: Human genomics research is growing rapidly. More effective methods are required for 
co-design and involving people, especially those from populations at greater risk of exploitation, 
such as those of inherently high interest to medical research. This case study documents how we 
worked with a large group of donor-conceived siblings who share the same sperm donor father, to 


explore how they might want to engage with and influence any future genomic research. 


Method: A participatory action research process was used to explore the views of a group of 18 
known donor-conceived siblings; who are part of a larger group of up to 1000 people who share the 
same biological father. The discussion explored views about how the group would like to be involved 
in future research. Five members participated in co-design; 12 completed a pre-discussion online 
survey; and six participated in an online discussion forum and evaluation survey. The online 


discussion was led by one facilitator, supported by the study team. 


Results: Co-design informed the research process. Participants reported enjoying the overall 
experience of the surveys and discussion forum, which were perceived as inclusive and flexible. Most 
participants’ views regarding the value of involvement in research changed during the process, and 
‘widened’ about who should be involved. Participants were supportive of future research being done 
with the siblings group. All who completed the final survey requested to remain part of the co- 
design process. Other themes in the online discussion included concerns about conflicting interests 
and a desire for research participation to improve the situation for people affected by assisted 
conception. The process informed later discussions in the sibling group about participating in a self- 


managed biobank and informed decision making about participating in genomics research. 


Conclusion: Findings from this study help inform how people from certain sub-populations should be 
involved in planning and defining their participation in genomic research; particularly those from 
populations at greater risk of exploitation or high-interest populations. This process provides a 
replicable and practical method of involving potential participants in co-designing genomics research 
using online discussions, with reported positive outcomes. Reporting this study using ‘Standardised 


data on initiatives (STARDIT)’ to report the process allows comparison with other studies. 
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Introduction 


Human genomics research involves defining sub-populations, measuring DNA changes within them 
and linking them to traits or outcomes in order to understand how DNA variation can contribute to 
human health and disease. The more genetically similar people are, the more likely it is that they will 
share the same DNA variations that contribute to a given trait such as wellness or disease. 
Therefore, historically human genomic research has focused on restricted populations who share 
common biological ancestry, including large families or founder populations, where less genetic 


variation has led to more clear links between DNA and disease.?°° 


Since the advent of affordable online ancestry DNA testing, increasing numbers of people are taking 
DNA tests to understand genetic ancestry (genealogy). However this DNA testing can sometimes 
result in returning unexpected information, including the revelation of being sperm donor- 
conceived.?” These genetic results can also immediately link people to a broader group of 


biologically related people who also share the same biological father.*%” 


Owing to the increasingly affordability of direct-to-consumer DNA testing?’’, a growing number of 
large groups of donor-conceived siblings are now being discovered, some with over 100 half- 
siblings.” In some cases, people from such sibling groups were conceived before regulation or 
clear legal oversight of assisted conception services. For example, in the UK, a government register 
of donors was proposed as early as 19497", but such a register was not established until 19917. By 
1958 the total number of donor conceived people in the UK was estimated to be 7,500, and 100,000 
in the United States. Some countries, including developed economies such as Japan, still have no 


clear legal regulation or oversight of human sperm and egg donation.100402403 


A historical and contemporary lack of regulation means that there may be many undiscovered large 
sub-populations of individuals who share a common sperm donor. Such sub-populations may 
provide new insights into medical, genetic, sociological and psychological studies, and therefore are 
of inherently high interest to research, including genomics research. However, such interest also 
makes them at greater risk of exploitation.” There are prior examples of other groups with 
shared biological ancestry having research conducted “on them” rather than in partnership “with 
them”.?4°>4°° The preferences of people within these sub-populations about how they would like to 
work with researchers conducting research with them and how they would like to be involved are 


established.? 
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Here, we involved people who know themselves to be members of a donor-conceived sibling group 
in co-designing this research. This case study documents a participatory action research process with 
members of one of the largest biologically-related (donor-conceived) family groups ever 


documented (hereafter ‘the sibling group’).222400407-409 


Members of the sibling group are all offspring of one sperm donor, the scientist Bertold Wiesner, a 
consulting biologist at the Royal Northern Hospital in the 1940s.*2° He may have fathered up to 
1,000 offspring between 1942 (or earlier) and 196774°%*°? through the medical practice of his wife 
Dr Mary Barton, despite a 1945 British Medical Journal paper where Barton and Wiesner stated they 
set an ‘arbitrary limit of 100 children for each donor’.*? This study represents the first of its kind 
done with this group of siblings. The 18 siblings known to each other at the time of this research 
project are the living members of the cohort who are both aware that they are part of a larger group 
of up to 1000 people who share the same biological father and have chosen to make ongoing 


contact with each other (hereafter ‘known siblings’). 


A formal database of who is descended from Wiesner is not known to exist. It is therefore possible 
that some descendants have lived their whole lives and then died, without knowing their biological 
father was Wiesner. The purpose of this research was to involve known siblings in online discussions 
about how they would like to be involved in future research, to better understand preferences of the 
group members in order to inform any future genomics research. More information about this 


sibling group and relevant contextual and historical information can be found in Additional File 1. 


We have also reported the participatory action research process and outcomes using the novel 
system ‘Standardised Data on Initiatives (STARDIT)’ , which is an open access data-sharing system to 
standardise the way that information about initiatives is reported. 1°>?37412 More detailed 


information about STARDIT can be found in the section about the Beta version.® 


Methods 


Study Design 


A participatory action research method was chosen for the study with co-design and reporting 
guided by a number of frameworks. 7>8%t23 ‘Participatory action research’ is an umbrella term which 


describes a number of related approaches, including forms of action research which embrace a 
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participatory philosophy and include ‘co-design’ and ‘co-production’ of research.***? It is a process 
where researchers, relevant stakeholders and sometimes the public “work together, sharing power 


and responsibility from the start to the end of the project”® 


, including knowledge generation and 
translation.®° The term ‘stakeholder’ means anyone who has a ‘stake’ in the research, in particular 
those with important knowledge, experiences, expertise or views that should be taken into 


account.?®30 


After one of the investigators (JN) discovered his biological grandfather was Bertold Wiesner, the 
study team invited a researcher who had previously worked with members of the sibling group (MC) 
to join the study team. The biological relationship of the investigator JN was then used as a starting 
point by the study team to work closely with both potential participants and experts in human 
research ethics from the La Trobe University to ensure the method was acceptable and that no one 
(including the study team) would be exposed to avoidable risk. Guidance on ethical involvement of 


potential participants in research design remains unclear! 


, in particular when it involves individuals 
conducting research with biological relatives.*** Accordingly it was decided by the study team to 
commence co-design only after ethics approval was obtained. Feedback from the co-design process 
was later integrated into the study design through multiple subsequent modifications to the initial 
research plan, with each modification being reviewed and approved by the Human Ethics 


Committee. 
The study had four stages: 1. Co-design; 2. Recruitment and pre survey completion; 3. Online 


discussion and post survey and 4. Analysis, with stakeholders involved in tasks at each stage (Figure 


7.1). 
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Figure 7.1: Stages of research 


Stage 1 sig Stage 2: Recruitment and survey Stage 3: Discussio Stage 4: Analysis 
July i October { October October October May 2019 December 


Preliminary Investigator team Invitation to participate Consent given, pre- 14 day online discussion with 6 _ Post-discussion Analysis 
research process established. shared with siblings via discussion survey shared, participants, with simultaneous SUTvey sent complete, 6 
agreed with co- Study co-designed online community details about online facilitator support group siblings 

investigators with sibling group (email group) discussion and learning checked 

representatives resources sent manuscript 


At each stage of the process, stakeholders were involved in the following tasks: 
y ty ae © = ny 


Checked wording Co-created learning 
resources 


However, reporting of such involvement is often inconsistent, of variable quality 
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, or the reporting 
frameworks themselves can have limitations.*° STARDIT was used to report involvement, as it 
overcomes some of the identified limitations of other reporting tools by allowing people to self- 
identify with multiple labels such as ‘researcher’ and ‘patient’, and by transparently allowing 
multiple stakeholders to be involved in reporting which stakeholders have completed which tasks." 
It has been proposed as a way of describing the ‘who’, ‘how’ and ‘what’ of research, and as a way to 


report two-way learning and ‘transformational learning’ amongst other impacts.” 


As the studies were designed to facilitate comparison in data, accordingly, some aspects were 
inflexible and thus more accurately described as ‘co-refined’ rather than ‘co-designed’. For example, 
questions in the pre- and post-surveys relating to ‘who should be involved in research’ and were 
designed to explore any changes in views on which stakeholders should be involved in which tasks 
during research, and were worded identically in each study in order to allow comparison with 
standardised data.’"* Participants could choose from the categories outlined in Figure 7.2, with a 
change in direction towards more people being involved labelled as ‘widening’, the inverse as 
‘narrowing’. The facilitator (MC) was also asked to complete a survey, which was informed by the 
Public Involvement Impact Assessment Framework Guidance (PiiAF)®” and the questions were 
informed by the GRIPP2 reporting checklist.’? The survey was identical to a survey used in other 
studies to facilitate data comparison.®’ All survey questions can be found in full in “Additional File 1 - 


Data and analysis’. 


Similarly questions about demographic information for the two studies were informed by the Genioz 


study to allow comparison with a wider dataset.” The choice of the software used in the online 
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discussion was also fixed owing to ethical constraints, including physical data location and ownership 


by participants. 
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Figure 7.2: Who should be involved in research? 


e any | Anyone who might Only people who 
ed member be indirectly affected are directly affected 
7 _ by the research by research 


Online methods of discussion are appropriate where a group of people is geographically dispersed 
and face to face discussion is impractical.**%° We therefore selected an online text-based discussion 
format, where people could log on at any time and contribute (asynchronous), including replying to 
other people’s comments (threaded discussion). This platform allowed flexibility compared to ‘real 
time’ (synchronous) discussion, especially regarding participants contributing from different time- 


”390 was installed on virtual machines 


zones. An open source software web application called ‘Loomio 
hosted by an Australian Government initiative called ‘Nectar Cloud’ .3*! Participants could securely 
log-in from anywhere in world and participants’ data was stored securely on servers physically 


located inside Australia. 


The online discussions were facilitated for 14 days and led by one facilitator (MC). The facilitator 
was trained in advance to use the online discussion platform, given learning resources, invited to be 
part of a community of practice of other facilitators running similar discussions’, and given support 
via telecommunications (by JN) to discuss any emerging issues. At the start of the online discussion, 
the facilitator asked participants to consider agreeing on boundaries relating to acceptable conduct, 
and invited members of the discussion to co-create a group agreement on conduct. The facilitator 
judged when to introduce new topics (depending on the engagement with each topic) with the 
recommend schedule below in Table 7.1 used as a template. The facilitated discussion period also 
included the provision of learning materials (short videos, infographics and information about 


research terms). These learning resources are included in ‘Additional File 1 - Data and analysis’. 
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Table 7.1: Discussion overview 


Agreeing boundaries Day one 
What do you understand by the word ‘research’? Day one 
What do you understand by the phrase ‘genomic research’? Day one 
Why do we do research? Day two 
Which aspects of any future research genomic research Day four 
should be influenced by the which groups of people? 

What methods do you think could be used to involve those Day six 
people in future genomics research? 

Do you have any ideas, thoughts or reflections that have not Day seven 
been shared yet? 


Discussion closed Day 14 


The learning resources provided during data collection drew on the outcomes of an analysis of 
educational materials by the Australian Genomics Health Alliance. 3? Resources were selected by 
the investigator team in partnership with the Australian Genomics Health Alliance and refined 
following feedback from the volunteers. Detailed information about which learning resources were 


shared at which stage of the study can be found in Additional File 1. 
Data analysis 


We collected and analysed both qualitative and quantitative data during the participatory action 
research process. Data sources included pre- and post- surveys; online text-based discussions; 
meeting notes; emails between the study team members; surveys of the study team; comments 
shared by two of the study team (JN, MC) during the online text-based discussion; and reflexive 
research diary entries of one member of study team (JN) (Table 7.2). The data collection and 
analysis was informed by a number of frameworks and approaches for conducting and reporting 
qualitative research*®?3%, and conducting?49194:417-420,196,157,159,174,176,178,368,370 and reporting case 
studies.1%*4? Participatory research processes, including stakeholder involvement, were informed by 
frameworks and best-practice for conducting”?®*"*!, reporting”? "2, and assessing involvement in 


research.®” 


Coding and thematic analysis of qualitative data was carried out by two authors (JN, MC) 


independently; the analysis then checked (triangulated) for validity by a third author (PL), and 
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participants were invited to review the analysis (‘member checked’), which is best practice for 
enhancing validity in qualitative methods.” In addition to quantitative analysis, each source was 
analysed using the method of thematic analysis, which involved stages including data mapping and 
familiarisation, transcription, coding, searching for themes, reviewing themes with study team 
members, labelling and summarising themes, and reporting the findings.*”® Participants were invited 
to review the analysis before publication in order to check whether they felt it reflected the research 


process accurately. They were also invited to share any further data via the STARDIT report.??” 


For the online discussion, we also used case study methodology to record and describe our 
experience of involving participants. This process is presented as an ‘instrumental’ case study, where 
the purpose is to understand the particular case, and attempt to provide data that could produce 
useful generalisations by using inferences from the data.**? The co-design of the case study was 
informed by best practices for enhancing validity and rigour in the case study 

methodology. 149156423,176-178,368,370,418-420 Iņ order to aid analysis and comparison with other case 
studies, we used ‘Standardised Data on Initiatives’ (STARDIT) Alpha Version to consistently map 
preferences for involvement and report ways people were involved in the participatory action 
research process.'*? Both the STARDIT report and the preference mapping can be found in the 
additional files. Further information about the case study method, data and both qualitative and 


quantitative analysis is provided in ‘Additional File 1 - Data and analysis’. 


Table 7.2: Summary of data analysis 


Data source description Analysis method 

Meetings — including meeting notes and relevant Qualitative (thematic analysis) 

documents 

Online survey — text data from open and closed Qualitative (thematic analysis, STARDIT-PM) 
questions Quantitative (number of responses) 


Online discussion — text data from a facilitated and Qualitative (thematic analysis, STARDIT-PM) 
moderated online discussion of both participants Quantitative (number of responses) 

and a separate one for the study team 

Study team surveys — responses by email Qualitative (thematic analysis, STARDIT-PM) 


Other data — reflexive research diaries, Qualitative (thematic analysis) 


Participants and recruitment 
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The selection of the particular shared ancestry population for this case study was informed by a 
number of factors which were appraised by the study team, including pragmatic considerations 


coupled with extensive consultation with professional ethics advisors and other experts.1”4 


In October 2018 one member of the sibling group forwarded an email invitation to the other 17 
known members to join the study. The invitation to participate contained a link to the participant 
information document; a plain English summary about the study; some learning resources about 
genomics research; and an informed consent form. We offered the choice of anonymity (through 
using pseudonyms or temporary email accounts) in survey completion and the online discussions to 


allow people to participate without disclosing sensitive or personal information. 


Those who gave consent were invited to complete the online pre-discussion survey and were then 
contacted by email directly by a member of the study team (MC), who shared information about 
joining the online discussion alongside relevant learning resources. This included a short 60 second 
online video about the study, giving information about the context and purpose, and a one-page 


infographic summary of a scoping review about involvement in genomics research. 
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Results 


Figure 7.3: Summary of the research process 
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VVOTT 2 pqa ers D WITN a groupo 


siblings who share the same donor father, 
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involving people and sharing power by shaping 
future genomic research being done with them 


Invitation to participate shared via 12 give consent and complete survey, 
email with half-siblings M ==> online discussion details sent M 


14 day online discussion with 
6 participants, 1 facilitator 


Standardised reporting of involvement in research 


jr Reporting and analysis c *Who was involved in which tasks? 
il STARDIT jnocvennarusen -Who is funding it and why? 
*What is the outcome or output? 
DAI Mo *How will data be shared? 
1 


is being developed by the D SCIENCE FOR ALL 
rnals to report research 2. © 


Stage 1: Co-design 


Concepts such as ‘co-design’ and ‘co-creation’ are used here to describe involving people in the 
respective tasks such as helping design the research project, or providing input during the creation 
of a learning resource. These concepts can be considered as part of participatory action research.*4 
Five members of the sibling group gave feedback during the co-design process (Stage 1), three of 
whom went on to participate in the study. Their input during the planning and co-design stage had 
clear positive impacts, particularly in improving educational resources and ensuring the pre- 
discussion survey used appropriate and acceptable wording to describe the sibling group, the 
members and the sperm-donor. During the co-design process it was agreed that the initial study 
should only involve the siblings, excluding any of their offspring. It was agreed during the co-design 
process that direct communication with the sibling group would be conducted by a biologically 
unrelated member of the study team (MC) once the study commenced, owing to the ethically novel 
situation of one of the investigators (JN). Figure 7.4 summarises how many people participated in 
the stages of the study. Of the 18 known living siblings in 2018, 14 participated in the co-design 


stages or the study. 


205 


Figure 7.4: Sibling group involvement and participation 


All known members of sibling 
group (as of October 2018) invited 
to participate in co-design (N=18) 


Potential participants involved in eb Two of the five involved in co-design 
co-design (N=5) did not participate in study (N=2) 


4 


Stage 2: Recruitment 


Siblings give consent to participate 
and complete survey (N=12) 


Siblings actively participate in Six of the 12 who completed survey 
online discussion (N=6) did not participate in discussions (N=6) 


Siblings participate in post- 
discussion online survey (N=6) 


Siblings request to stay involved in 
the subsequent participatory 
action research process (N=6) 


Siblings involved in giving 
feedback on article before 
publishing (N=6) 


Stage 2: Recruitment and surveys 


Of the 18 members of the donor sibling group, 12 gave consent to participate and completed the 
pre-discussion survey of whom six participated in the online discussion. All six completed the follow- 
up survey in May 2019 and shared consistent identifying data at all stages, including the initial 
survey, the online discussion and the follow-up survey, allowing a comparison of results throughout 
the process. 

Of the 12, seven were female and five were male. All stated that they were comfortable describing 
other descendants of their biological father as ‘half-siblings’. Self-reported educational attainment 
was mostly ‘degree (bachelors), diploma or post-graduate’, with one participant having professional 
experience in medicine and genomics. All were aged between 50-74, with most living in the UK. 
Detailed demographic information can be found in ’Additional File 1 - Data and analysis’. Most 


agreed with the statement that they have a shared interest in discussing future research which 
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might affect them, including genomic research. More information about participant demographics 
can be found in Additional File 1. 

From the six participants who completed both the pre-discussion and follow-up surveys, a total of 54 
responses were given. Of these, 35% showed a change towards ‘widening’ involvement (N=19/54), 
8% ‘narrowed’ (N=8/54) and half stayed the same (N=27/54). More detailed data is provided in 
Additional File 1. 


Stage 3: Online discussions 


Six people participated in the online discussion in October 2019. Only one chose to use a pseudonym 
in the online discussion, so the study team could identify them. Some of those who gave consent to 
participate did not make comments but logged-in multiple times and read comments. Table 7.3 
shows how many comments were made by each participant in the online discussion. The themes 
where most participants shared views are summarised in Table 7.4. The most discussed themes 
included the unique nature of the sibling group, that anyone should be involved in research, 
questioning who decides ethical oversight in research, questioning research for profit and sharing 


concerns about genomics research being used for political purposes. 


Table 7.3: Number of comments in online discussion 


P2 12 
P4 14 
P5 42 
P6 15 
P7 9 

P9 13 


Facilitator (MC) 65 


Table 7.4: Quantitative summary of themes 


Theme Number of 


participants 


Anyone should be involved in research 6 
Research with sibling group is unique and complex but important 6 
Those affected by research should be involved 6 
Research for profit and ‘bullying’ by ‘big pharma’ 6 
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Theme Number of 
participants 

Who decides who decides what is ethical? 

Concerns about genomics research being used for political purposes 

Finding out they are part of sibling group has been a positive experience 

View on topics for research 

Participants reported changed views and perspectives as a result of participating 

Desire to improve situation for people affected by assisted conception 

Interested in learning what other siblings think and discuss issues 

Concerns about control of knowledge and data 

Questioning which groups should have ‘equal influence’? 

Questioning eugenic attitudes to genomic variations 

Views on participation in genomics research 

Participants learned about genomics 

Motivation for participation to help researchers and sibling group 

Uncertainty about what they can offer but happy to help 

What is the purpose of research? 


Experts should be involved (over seen by ethics boards) 


N N NY NY NY Ww Ww Ww WwW WwW WwW WwW FP FP FP A A 


Developments in genomics have significant implications for society 
Stage 4: Analysis 


Participant feedback in the post-discussion survey was positive, including that the experience was 
‘interesting’, and that they ‘enjoyed thinking about the questions posed and reading the responses 
of others’ and the ‘perceptive comments’ of the facilitator [P7] [P4]. Two reported experiencing 
some usability issues with the online platform. All who completed the survey opted to stay involved 


in the next stages of the participatory action research process. 


Table 7.5: Summary of pre and post survey responses 


What made you decide to respond Participants wanted an ‘opportunity to be involved in research’ 
to our invitation to participate in [P3], wanted to ‘learn more’ [P7] and regarded involvement as a 
this project? 12 participants ‘civic duty’ [P10]. Others stated this study may be useful to 


responded (pre-discussion) ‘future donor offspring’ [P11] and one participant stated 


208 


What do you hope to get out of 
participating in this discussion? Do 
you have any specific 
expectations? 8 participants 


responded (pre-discussion) 


Do you have any ideas about how 
the different people could 
influence future research? 7 
participants responded pre- 


discussion and 3 post-discussion 


Is there anything in particular you 
liked or thought was helpful about 
how the discussion was 
conducted? 5 participants 
responded (post-discussion) 

Is there anything you didn’t like, 
thought was unhelpful or could 
have been improved about how 
the discussion was conducted? 5 
participants responded (post- 
discussion) 

Did you have any expectations 
from participating in this research 
that were met or not met? 6 
participants responded (post- 


discussion) 
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‘curiosity’ was a reason for participation. Four participants noted 
a familial connection to a member of the study team (JN). 
Participants were ‘interested in hearing what their half siblings 
think’ [P4] and wanted an ‘opportunity to discuss’ and ‘think 
through the issues involved’ [P4] [P6]. One participant said they 
wanted ‘the satisfaction of knowing that | may have contributed’ 
to the study [P10]. Another wanted ‘to be useful to the 
researchers’ [P11]. One participant noted an expectation of 
anonymity while participating. 

Five participants said ‘everyone should be involved in research’, 
with one adding ‘not just scientists and researchers’ [P5-pre]. 
One participant said ‘anyone with an opinion’ should be involved 
[P3-pre]. Another stated ‘researchers and those who are affected 
by what is being researched’ [P4-pre]. One participant stated 
‘People who know their subject but do not have hidden motives 
or agendas’ should be involved [P2-pre]. One participant said the 
answer depended on ‘what kind of research it is’ [P7-pre] and 
one mentioned ‘ongoing discussions’ using online tools [P5-post] 
One participant stated they ‘liked and appreciated the 
opportunity to participate’ [P6]. One participant stated the 
process ‘seemed to work well’ [P7]. One participant added ʻI 
think it is commendable that there is a concern about 
participatory research’ [P9] 

Two participants reported finding the platform ‘complicated’ and 
problematic [P5]. Two participants stated they would have liked 


‘more time’ for the process [P4]. 


Four participants stated their expectations were met. One 
responded that they ‘found some of the questions very complex 


and had difficulty answering them’ [P5]. 


Outcomes from the process 


There were 8 outcomes reported from this process, which are summarised in Table 7.6, with 


additional information available in the accompanying STARDIT report (Additional File 2). 


Table 7.6: Summary of outcomes from the process 


Outcome 
1. Improved understanding of 
genomics informed participation 


in future research 


2: Learning resources useful 


3: Changed views and perspectives 


as a result of participation 


4: Participants asked to stay 


involved in the research 


5: Participants enjoyed the online 


discussions 
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Summary and learning point 

Participants reported their understanding about genomics 
research increased as a result of participating in the study. 
Learning from this process informed subsequent discussions 
in the sibling group about participation in research, 
including a proposed self-managed biobank. 

Participants reported finding the information resources and 
videos useful. Learning point: Creating learning resources in 
multiple formats (hyper-text, infographic summaries, videos 
with subtitles animations) will ensure that information is 
more useful for people with neuro-diverse learning needs. 
Participants reported their views and perspectives changed 
as a result of participating. Learning point: Online 
discussion facilitated collaborative learning and the changed 
views of participants can be viewed as an impact of 
‘transformative learning’ 127167446 

All participants who completed the follow-up survey opted 
to stay involved in the research process. Learning point: 
Participants demonstrated ‘critical reflexivity’, a stage of 
participatory action research which asks people involved to 
reflect on their views and the causes of problems and to be 
involved in exploring any solutions and the actions that 
people can take to bring about improvements.126227132 
Participants stated the experience of participating was 
‘interesting’ and they ‘enjoyed’ it [P7] [P4], despite some 
usability barriers. Learning point: Some participants stated 
they preferred online discussions over face to face 
discussion or interviews, which highlights the importance of 


mapping potential participants’ preferences when co- 


designing involvement, to ensure research methods meet 
the needs of participants. 

6: Improved understanding of how Participants reported improved understanding of how to 

to get involved in research get involved in research; this helped inform decision making 
for individuals when invitations were sent to members of 
the group to participate in genomics research after this 
study and unconnected to this study.”?” Learning point: Pre 
and post discussion surveys and standardised reporting 
(STARDIT-PM) are useful tools for mapping changes in 
understanding and views.?2°192 

7: Co-design changed study design Feedback from participants resulted in changes to the study 
design including improving language used in recruitment 
and learning resources. Learning point: Involving 
participants in helping co-create learning resources can 
improve them. 

8: Method for future research co- Participants stated that the methods used in this process 

design established could be helpful when co-designing future stages of 
proposed genomic research with the sibling group. Learning 
point: STARDIT can be used to map preferences and 
impacts from future co-design processes.’ Learning from 
this process is relevant to sub-populations where people 
share recent ancestry such as some Indigenous 
populations”*”, and sub-populations of people affected by 


rare diseases.” 
Participant views about genomic research and involvement 


Participants demonstrated improved understanding of the difference between participation in 
research and involvement in research by the end of the discussion, although initially some were 
confused by the distinction. One stated 'I am a strong supporter of patient involvement in medical 
care' and that 'involving members of the public’ in genomic research was important in order to 'have 
their views, reactions, interpretations, questions, concerns sought, interacted with, and considered’ 
[P11]. 

All six participants of the online discussion thought that anyone should be involved in research, with 
one saying 'everyone should have a voice not just scientists and researchers' [P5]. Another added 


that it is a 'good idea to involve research subjects in formulating the research questions' [P10]. 
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Methods of involving people were discussed in detail with a number of options explored. 
Participants explored ideas around the purpose of research and one stated that research 
participants should be involved in 'agreeing purpose, parameters and methods' [P7]. All expressed 
concern about research for profit and those with financial interests influencing research without the 
public being involved. 

One participant noted that being ‘highly educated’ was an enabler for involvement as was having a 
‘bit of time on their hands' [P4]. Another noted they didn't feel 'qualified' to ‘comment on aspects of 
science itself! but felt ‘strongly’ that they should be involved in ethical decisions and sharing personal 
experiences to help inform research [P5]. 

One participant asked 'there will be many interested groups so which ones will be listened to?' [P4]. 
Another noted they felt that certain pharmaceutical companies were responsible for ‘bullying’, 
contributing to ‘disinformation; ignorance and inflexibility of medical and scientific professions’ 


[P12]. 
Participant views about proposed research with sibling group 


Participants were supportive of future genomics research with the sibling group. One participant 
stated it would be ‘worth the effort’, and another stated they ‘wholeheartedly support the 
involvement of the next generation’ in any future research with the sibling group and noted any 
study design should ensure new siblings and their offspring should be involved to ensure they can 
‘become part of the research’ [P9] [P5]. 

In reference to future research with the sibling group one participant stated that ideally 'we would 
be able to exert control over the use' of data [P7]. Conversely, one participant stated in the follow- 
up survey that they ‘could not at all care whether my genomics are public or not. | do not see that 
my genome is a matter for privacy concerns’ and recognised that others may feel differently [P9]. 
Participants spontaneously shared their views about what possible areas of research topics could be 
explored in studies they could participate in and how these could be conducted. These included 
‘mental health' [P6] and pharmacogenomics [P4], with non-health related topics including 'career 
choices' and hobbies [P4]. One participant felt they should be involved in 'seeking answers to old, or 


not yet thought of questions' and ‘looking beyond the known into a murky unknown! [P6]. 
Stakeholder views about the process 


Participants reported their motivation for participating in this research process was altruistic, to help 
researchers and the wider public. Some also recognised that their participation might directly 
benefit members of the sibling group itself. One participant also stated ‘it is commendable’ that 


‘participatory research’ was being used, in reference to the research methods used by the study 
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team [P9]. While online discussions were perceived as having advantages and ‘worked well’[P7], two 


participants reported usability issues with the online platform. 


Four participants identified specific things about the way this study was conducted that enabled 
their involvement. One said the entire process was ‘assiduous’ and that the ‘intent of this project’ 
was ‘obviously thoughtful and interesting’ [P9]. One participant said the ‘system seemed to work 
well’ [P7]. Another added that being used to online platforms like Loomio, or having previous 
experience of similar platforms and ‘used to’ that way of communicating might facilitate 


involvement using such a communication mode [P5]. 


Four participants reported specific things about the way this study was conducted that were barriers 
to their involvement. The pace of the discussions was mentioned as moving ‘too quickly’ with 
another adding ‘more time’ was needed and that the study team should ‘reconsider the pace of the 
research’ [P7] [P4] [P5]. A discussion about creating boundaries in the discussion revealed that some 
participants felt that they should avoid ‘topics which might trigger emotions which are stressful’ 
whereas others thought this could be viewed as ‘restrictive, even censorious’ [P7]. 

The study team noted a ‘critical mass’ effect in online discussions, with the pace of comments 
seemingly affected by the number of posts. One study team member noted the difficulty in 
achieving ‘the balance of being prescriptive’ (for consistency) and giving freedom to facilitators to 
initiate discussions and follow emergent themes. Support for the facilitator by the study team was 
identified as an important enabler of the research process by the facilitator. Further barriers and 


enablers are summarised in the accompanying STARDIT report (Additional File 2). 
Discussion 


In this study we used a participatory action research process to explore the views of a group of 18 
known donor-conceived siblings, with participants reporting enjoying the overall experience of the 
surveys and discussion forum. Online text-based discussion forums were reported as an inclusive 
way of involving people which was more flexible for communities which exist across time zones. 
Participants reported it also gave them more time to reflect on answers, learn collectively as a group 
and provided the freedom to ask questions and share ideas throughout the process. Participants 
reported that the participatory process described here appears to have resulted in members of the 


sibling group feeling they will have more influence over research done with them??’. 
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Co-designing the study ensured it was more likely to meet the needs of potential participants. 
Involving participants in co-designing the research process resulted in a number of improvements to 
the study design, including improving language used in recruitment and learning resources and co- 
creating acceptable online discussion boundaries. The process improved participants’ understanding 
about genomics and research. However, data from participants indicates that increasing the time 


allowed for any future discussions would help ensure the process does not move ‘too quickly’ [P7]. 


Participation in the process led to eight outcomes, including participants ‘widening’ their views 
about who should be involved in research to include more people. Participants reported changed 
views about the value of involvement and an improved understanding about how to be involved in 
genomics research. Some participants reported via the co-authored STARDIT report that learning 
from this process informed subsequent discussions in the sibling group about participation in 
research, including a proposed self-managed biobank and helping them make informed decisions 


about participating in genomics research. 


During the online discussion, the facilitator made significantly more comments than any one 
participant, reflecting the level of work and engagement required to facilitate a discussion. The 
‘critical mass’ effect of a certain number of posts in a discussion encouraging others to post aligns 
with findings from other studies which have explored participants’ hesitancy in posting in online 
discussion forums**. Only one participant used a pseudonym during online discussions, and this may 
reflect that the other group members felt comfortable sharing views with others in the group and 
trusted the security and privacy of the platform, however it is unclear why some participants logged 


in but did not comment. 


Throughout the process, the study team was faced with ambiguous policies for the ethical 
involvement of people in co-designing participatory research, which hamper the degree of control 
that potential participants had in the research process. Members of the study team reported they 
felt that limitations in the ethics process affected the extent to which the sibling group could be 
involved in the participatory action research process. This is reported in more detail in the 
accompanying STARDIT report (Additional File 2). Additionally, an unplanned delay of 9 months in 
collecting follow-up data (related to ethics processes) may have affected the quality of data 
collected post-discussion, with one participant adding ‘given the lapse in time, | cannot answer’ in 


response to a question about their experience of participating in the online discussion [P6]. 
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Further clarity from ethics committees and relevant organisations about the ethics of participatory 
action research will enhance power sharing at this crucial stage of research, with standardised 
reporting of data allowing direct comparison of ethical methods of involvement. One participant’s 
suggestion that the sibling group should form their own ethics committee to guide future research 
reinforces the importance of transparent ways of sharing power and control in genomics research 
[P6], in particular for sub-populations more likely to be exploited. This includes other sub- 
populations where people share recent ancestry such as some Indigenous populations??>*™; 
populations of people affected by rare diseases caused by similar genomic variations”; and other 


sub-populations where there is a perceived power-imbalance between researchers and potential 


participants. 


While the focus of the online discussion was involvement in genomics research, the discussion 
entered additional areas. This reflected the interconnected nature of the subject, including the 
ethical, legal and social implications of all genomics research. The discussions thus served as an 
exploratory focus group, mapping participants’ views in these diverse areas. For example, four 
participants spontaneously raised the ‘ugly’ issue of eugenics and eugenic attitudes to genomic 
variations, including the distinction between perceived disability and lived-experience of having 
certain genomics variations [P6]. Noting the history of eugenics, participants expressed fears that 
genomics research would continue to be used to reinforce racism and assist with genocide. One 
participant cited the well-documented historical precedent of a large technology company being 
complicit in enabling regimes to carry out negative eugenics policies*”’, and also raised concerns 


about contemporary and future ‘misuse’ of genomic data ‘for immigration’ [P6]. 


The complex ethical, legal and social implications of genomics research reinforce the importance of 
further research to evaluate data from discussion methods such as the ones described in this article. 
Evidence-informed methods are urgently required in order to inform education, debate and develop 


international consensus on the ethical use of data from genomics research. 


Strengths 


Measuring outcomes such as ‘transformative learning’ can be challenging, but is an essential 
requirement for understanding processes such as involving stakeholders in research and knowledge 
translation!?”167416, The process followed in this study approached such challenges by collecting 


standardised baseline and follow-up data about views. The findings from these measures suggested 
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that involving people in online discussions about involvement in research can change their views 
about who should be involved in research, including ‘widening’ such views towards support for the 
inclusion of a wider category of stakeholders. Similarly, using STARDIT for standardised preference 
mapping and reporting of involvement meant outcomes from the process could be mapped more 


precisely?26192 


, including outcomes beyond the date of the initial data collection. Participants 
reported their views and perspectives changed as a result of their participation, suggesting that 
online discussion facilitated collaborative learning; changes in views; and ‘transformative 


learning’ (127,167,416 


Participants reported their understanding about genomics research increased as a result of 
participating in the study. Data from the STARDIT report completed at the end of evaluation stage of 
the research process indicated that learning from being involved in online discussions informed 
subsequent discussions in the sibling group about participation in research, including a proposed co- 
managed biobank.2”’ Since participating in this study, some known siblings have received invitations 
to participate in genomics research and share their genomes with researchers unconnected to this 
study. The research project described in this article helped inform their decisions to participate’*’, 
and how they would like to be involved in shaping the future of other research with the group. In 
addition, the sibling group has also begun discussions within the group about future research with 


the group, including a self-managed biobank, with discussion informed by learning from this 


process.” 


Involving potential participants in the co-design of any future research is essential to ensure it is 
appropriate and acceptable. The importance of using STARDIT-PM for standardised preference 
mapping when co-designing involvement was also demonstrated by participants stating their 
preference for online discussion methods over face-to-face discussion or interviews. Such 


preference mapping helped ensure the research design met the needs of potential participants. 


Providing an ‘updateable’ way to report ongoing impacts and outcomes is impossible or, at best, 
impractical with traditional academic publishing. Since starting the study, the number of known 
living relatives has grown to 46 half-siblings and 24 half-cousins, owing to previously unknown 
siblings and cousins taking direct-to-consumer DNA tests. The updatable ‘living’ report provides a 
way for these additional siblings to integrate the views and preferences into future STARDIT reports, 


and report any ongoing impacts or outcomes. 
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While this study was co-designed and conducted before the COVID-19 pandemic, the learning 
outcomes from the process (summarised in Table 6) have relevance to research disciplines beyond 
genomics. The methods of involving people aged 50-74 in an online discussion described here now 
have an unexpected relevance to many disciplines. As research projects around the world seek to 
involve people online in novel ways, using a system such as STARDIT to report and evaluate such 
methods (including co-design) in a standardised and updateable way that works across languages is 
more important than ever.*” In particular, as more people become aware of the importance of 
storing data according to the preferences of research participants, hosting online discussions on 
platforms where data is stored according to these preferences is vital to ethical research conduct.®’ 
The detail and transparency of the methods described by this article and the accompanying STARDIT 
report provide a repeatable method for co-designing such discussions, using free and open-source 


software. 
Limitations 


While 14 of the 18 known siblings were involved in either co-design or as participants, only 12 
siblings gave consent to join the study, and only 6 of these participants made comments in the 
online discussion. Some of those who did not make comments logged-in multiple times and read 
comments. While the follow-up survey attempted to capture the views of all 12, only the 6 active 
members responded, so it is hard to understand the behaviour patterns of the 6 members who did 
not comment. The relatively small starting cohort (18) for this study and a smaller number of active 
participants (6) means that although there is useful learning from our findings, their statistical 
significance can only be enhanced by standardised data sharing that can then combine with results 
with other studies. 

As the study was designed in parallel with other studies, some aspects of the study were inflexible 
during the co-design process (such as the specific open-source discussion platform that was used). 
Some aspects could more accurately be described as ‘co-refined’ rather than ‘co-designed’, thus 
limiting some areas which could be influenced by the co-design process. Using STARDIT to report 
which stakeholders did which tasks has attempted to overcome subjective distinctions between such 


terms. 
In the time between the start of this research process and the publication date of this thesis 
(December 2021), there are now 46 half-siblings and 24 half-cousins known. Ethical constraints 


limited recruitment of newly discovered siblings. Further involvement to understand views and 
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preferences about research with the now much larger sibling group would help ensure any future 


participatory research meets the needs and expectations of the group. 
Conclusions 


The process described here provides a replicable and practical method of involving potential 
participants in co-designing genomics research using online discussions, with reported positive 
outcomes. Co-designing the study ensured it was more likely to meet the needs of potential 
participants and resulted in improvements to the study design. Reporting this study using 
‘Standardised Data on Initiatives (STARDIT)’ to report preferences, plan and report involvement, 
evaluate participatory research methods and report ‘updateable’ outcomes allows ongoing 
comparison with other studies. Such reporting facilitates learning from this case study and 
contributes to data to inform evidence-based decision making when planning future research. 
Learning from this study contributes to the current evidence-base used to inform future ways of 
involving people in genomics research. Such evidence can be applied in the context of research such 
as self-managed biobanks for sub-populations more likely to be exploited and other sub-populations 


where there is a perceived power-imbalance between researchers and potential participants. 


This is the end of the extract from the article ‘Co-designing genomics research with a large group of 


donor-conceived siblings’: https://doi.org/10.1186/s40900-021-00325-7 
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STARDIT MICRO Report: Co-designing genomics 


research with donor conceived siblings 


This STARDIT MICRO report contains the minimum data required for a Standardised Data on 


Initiatives Alpha version report (STARDIT Alpha).*°? A full version of this STARDIT report can be found 


in the supplementary resources of the article available here: 


https://doi.org/10.1186/s40900-021-00325-7 


The full prospective STARDIT report for this article can be found in the Appendices under the 


heading ‘Standardised Data on Initiatives (STARDIT) report — Alpha Version: Co-designing genomics 


research with a large group of donor-conceived siblings’ with a link to the online ‘living’ Beta version 


of the STARDIT report available in the references.7?’ 


Name 


STARDIT MICRO Report: Co-designing genomics research with donor 


conceived siblings 


Involvement 


Who was involved 


Group 1: Academic research investigators (Jack Nunn, Marilyn Crawshaw 
and Paul Lacaze) 

Group 2: Members of the sibling group who gave feedback during the co- 
design stage (including but not limited to David Gollancz and Michael 
Bywater) 

Group 3: Members of the sibling group who participated in the research 
and gave feedback as part of the co-design process, the manuscript 
checking stage or contributed data to the STARDIT report (including but 
not limited to Shirley Brailey, Barbara Nunn, Adrianne Smith and Barry 


Stevens) 


Specific tasks of this person or 
group (list as many as possible) 
— including any information 
about why certain people were 
included or excluded in certain 


tasks 


Group 1: Involved in co-designing every stage of the process, analysing 
data 

Group 2: Members of the sibling group were involved in refining wording 
of participant information, sharing views and advice about the process, 
proof-reading documents, providing feedback on surveys, analysing data, 


informing planning, and providing feedback on planned online discussions. 
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Group 3: Participants were also involved in checking the content of 
Genetics Society UK podcast*”®, with the recording shared with all 
participants before dissemination to ask them to check the content was 
accurate and acceptable. 

Group 2 and 3: Participants were sent the article and additional files to 


check the analysis and content and were invited to be authors of the 


STARDIT report. 
What was the outcome or Improved participant information resources, improved wording that was 
output of the involvement of culturally appropriate (using terminology preferred by the sibling group to 


these people? What changed as | describe biological relations), improved online discussion, improved 


a result of involving people? learning resources for participants, improved co-design process. 


One investigator (Jack Nunn) is biologically related to participants from the 


sibling group, with one being his mother and all being half-aunts or uncles. 


N/A 

Data 

Who is the data from this It will be published open access in peer reviewed journals with identifying 

intervention shared with? information removed in order to anonymise it as much as possible. 

How is it stored and hosted? It will be shared on a public domain repository. 

Who is analysing the data? Group 1: The study team described above 
Group 2: participants were invited to review the analysis and give feedback 
to ensure they felt it reflected their experience of the process 

How is information about this It will be published in an open access journal 

data disseminated? It will be shared with participants of the research and also other members 
of the sibling group who have joined it since the study commenced. 
Learning from this process has been presented at conferences, and will be 
shared on social media and through other channels. Preliminary learning 
was shared in a UK Genetics Society podcast”. 

How is the data FAIR°°? Shared in the public domain licensed under a Creative Commons. 
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What new knowledge has been 
generated? (if appropriate, 
include effect size, relevant 


statistics and level or evidence) 


Involving participants in co-designing the research process resulted in a 


number of changes to the study design, including improving language used 


in recruitment and learning resources 


The process of involving people can be viewed as a learning experience for 


both the participants involved and study team members. The process 


changed participants’ views about who should be involved, which can be 


viewed as an impact of ‘transformative learning’. 


Outcomes - Describe how the 
learning or knowledge 
generated from this initiative 


has or will be used 


1. 
2. 


Co-design changed study design 

The process improved participants understanding about genomics 
and research 

Learning from this process informed subsequent discussions in the 
sibling group about participation in research 

Participants reported finding the learning resources useful 
Participants changed views and perspectives about genomics 
research as a result of participating. 

Participants asked to stay involved in the research. 


Participants enjoyed the online discussions. 


Ethics approval and consent to participate 


The La Trobe University Human Ethics Committee approved this study. 


Project number: HEC18256 


Project Title: Genomics Research and Involving People: Exploring the views and perspectives of 


people with shared ancestry about being involved in genomics research 
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Chapter 8 — Involving Australian 
Indigenous peoples in precision 
medicine 


About this chapter 


The UN has raised human rights concerns about Australian Aboriginal and Torres Strait Islander 
peoples’ (hereafter ‘Aboriginal’ peoples) access to health.” In this chapter, | present a reflective 


h, 163,428 a nd 


case study in association with Sydney University’s Poche Centre for Indigenous Healt 
their proposed ‘Aboriginal Precision Medicine Project’. The project aimed to involve local community 
members in its design, data collection and management, in line with international best practice in 
community-controlled research. My participation included working with Aboriginal people in a 


remote community in Australia to co-design a protocol for genomics research. 


This chapter summarises the background, methods, results, reflections and conclusions from my 

work, including learning and reflection points in the Discussion section. Unlike the preceding three 
results chapters, this chapter as a whole is not published in peer-reviewed journals as a single case 
study. It is an overview of multiple different activities, represented in part by three publications in 


the area of genomics research and involving Indigenous peoples, which | authored or co-authored. 
For clarity, where content from published articles is used, this is denoted with italics. 


A STARDIT MICRO report is included at the end of this chapter, with the full prospective STARDIT 


report in the Appendices under the heading ‘Prospective STARDIT Report: A Pathway to precision 
medicine for Aboriginal Australians’, with a link to the online ‘living’ STARDIT report in the 


references.*7° 
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Chapter summary 


Background 


Precision medicine, involving knowledge of a person’s genome, offers a new way to improve the 
prevention, diagnosis and treatment of diseases. If implemented without consideration of equity 
issues, it could increase health inequalities. To improve the quality of healthcare and precision 
medicine for Indigenous peoples worldwide, improved ways of involving people that are culturally 


competent and safe are urgently needed. 
Method 


A reflective case study method is used here to describe my work with the Poche Centre for 
Indigenous Health at the University of Sydney, helping to advise on how the proposed ‘Aboriginal 
Precision Medicine Project’ can involve local community members in its design, data collection and 
management. The method and structure of this reflective case study is informed by the ‘reflection 


cycle’, and other relevant reflective case studies. 
Results 


Results from the participatory action research process included learning from the co-design process 
which informed the protocol creation, and the co-creation of the published protocol itself in 2021. In 
parallel to this participatory process, work with the National Centre for Indigenous Genomics (NCIG) 
on an article is also summarised, as discussions with staff from the NCIG informed Poche’s 
participatory action research process, and learning from this process also informed the article led by 


the NCIG. 
Discussion and reflections 


Co-defining appropriate and acceptable terminology is a crucial stage of the co-design process. 
STARDIT has been designed to align with models of self-identification, and can be used to report 
how people have been involved in an initiative, with people able to chose any labels they self- 
identify with, including ‘Aboriginal’, ‘Indigenous’ or ‘community member’. This removes the 


requirement (and the associated power) of researchers assigning such terms to people. 


The published protocol demonstrates how such co-design methods can be reported, helping ensure 


future ethical oversight can be managed in partnership with Aboriginal people. STARDIT can be used 
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to describe who was involved in co-designing every aspect of the research, and who was involved in 


making ethical decisions, providing more transparency to the entire process of ethical oversight. 
Conclusion 


The work described in this chapter provides a clear methodology for both planning and reporting co- 
design methods in genomic research with Aboriginal people. STARDIT can be used to create public 
records of every stage of the co-design process, including data about who was involved, how, and in 
which tasks. Information about the co-design method, the data analysis methods, reported enablers 
and barriers for involvement, and any other impacts and outcomes from the co-design process can 
also be reported. Such data improves transparency of terms such as ‘co-design’ and allows improved 


scrutiny and ethical oversight, including from potential or current research participants. 
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Introduction 


This section is adapted from an invited article | authored, published in ‘Health Voices’, the Journal of 
the Consumers Health Forum of Australia. Titled ‘Reducing health inequalities by involving indigenous 
people in genomics research’, the article was reviewed by the editorial board prior to publication in 


2019. ? 
Background 


Precision medicine, involving knowledge of a person’s genome, offers a new way to improve the 
prevention, diagnosis and treatment of diseases. If implemented without consideration of equity 
issues, it could increase health inequalities. To improve the quality of healthcare and precision 
medicine for Indigenous peoples worldwide, improved ways of involving people that are culturally 


competent and safe are urgently needed.*2°.434 
Health inequality for Indigenous Australians 


The UN states that Indigenous people have “social, cultural, economic and political characteristics 
that are distinct from those of the dominant societies in which they live”, and that “special measures 
are required to protect their rights”.°* The UN has raised human rights concerns about Australian 
Aboriginal and Torres Strait Islander peoples’ (hereafter ‘Aboriginal’ peoples) access to health.*”” The 
World Health Organization (WHO) and many other bodies have reported that Aboriginal peoples 
have lower than average life expectancies.*** A 10-year gap in life expectancy for Aboriginal peoples 
compared with other Australians has remained largely unchanged between 2001 and 2015, despite 
attempts to improve health outcomes. The nation’s target to close the gap in life expectancy by 
2031 “is not on track”.434 Depending on how it is implemented among Aboriginal peoples, precision 


medicine has potential to help close or widen that gap. 
What is precision medicine? 


Precision medicine refers to specialised and personal diagnosis and treatment of people. It requires 
knowledge of an individual’s DNA (their genome) in order to make comparisons with what we 
already know about how genetic variations contribute to disease. There are significant differences in 
the type, distribution and frequency of variations in the genomes of human populations across the 
world.*° For precision medicine to be effective, variations in the genome should be interpreted in 
the context of ‘reference genomes’ from appropriate ancestry groups.** These reference genomes 
are developed by combining the genomes of different individuals.?”° The more genetically similar 


people are to the reference genome, the more helpful precision medicine will be. 


225 


Why is there potential for further inequality in precision medicine? 


There is a paucity of genomic data about Aboriginal ancestry groups as there are currently no 
reference genomes from these populations.?”° This lack of genomic data for specific ancestry groups 
may increase existing health inequalities in Aboriginal communities, as these populations will not be 


able to use precision medicine as effectively as others.*%° 
How can inequality be reduced? 


To improve the quality of healthcare and precision medicine for Aboriginal peoples in Australia, it is 
necessary to improve the research done with these communities. Building trust and working in 
partnership at all stages of the research process can minimise the risks of group and cultural harms 
associated with genomic research.*?” However, some researchers see the complexities of working 
with Aboriginal communities to co-design and govern data collection, and use it in inclusive, 
culturally appropriate ways, as a barrier to research. The ‘unacceptably high level of disadvantage’ 
and the low-quality of healthcare has prompted the UN to recommend that Australia “reset the 
relationship with Aboriginal people based on genuine consultation, engagement and partnership” .*?” 
Improved ways of involving people that are culturally appropriate and genuinely share power are 


urgently needed.427430438-441 


A strategic priority of the Australian ‘National Health Genomics Policy Framework’ is to “promote 
culturally safe and appropriate data collection and sharing” for Aboriginal peoples. The Framework 
also states that the value of genomic research “needs to be compared to the range of alternative 
options in terms of its ability to deliver health benefits, its capacity for harm and the costs of 
testing.”“4? While existing unequal access to quality primary health care and other basic needs must 
be reduced,**? to avoid amplifying inequalities further, the benefits of genomic medicine must be 
delivered equitably to Aboriginal communities, alongside other healthcare interventions. 

The WHO has highlighted the importance of creating the conditions that enable people to take 
control of their lives.“ In the USA, the ‘All of Us’ ‘Precision Medicine Initiative’ showed that 
“precision engagement” is required in genomics research.” If the Australian Government is 
“committed to working in genuine partnership” with Aboriginal peoples,** this must be 
demonstrated by adequate funding for working in partnership, including supporting communities to 
evaluate their involvement and creating an evidence base for best practice. If lessons from the 
Belmont Report on ethical principles for research involving human subjects are to be heeded,*° 
vigilance is needed from the public, policy makers and researchers to ensure that the concept of 


‘involvement’ is not reduced to a method of improving recruitment and participation in research 
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which does not reflect the health priorities of those participating.**© Everyone should be involved in 


shaping the future of genomic research, and everyone should have access to the benefits of this new 


knowledge. 


This is the end of the content adapted from the article ‘Reducing health inequalities by involving 


indigenous people in genomics research’.” 
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Aboriginal Precision Medicine Project 


The proposed ‘Aboriginal Precision Medicine Project’ (hereafter ‘the Precision Medicine project’) will 
be led by the Poche Centre for Indigenous Health at the University of Sydney (hereafter the Poche 
Centre), whose staff are 62% Aboriginal. The Centre works to influence genuine change in public 
policy and identify approaches to improving Aboriginal and Torres Strait Islander health. As part of 
my doctoral research, | was invited to volunteer with the Poche Centre, helping to advise on how the 
Precision Medicine project can involve local community members in its design, data collection and 
management, in line with international best practice in community-controlled research. 

The Centre has been working with some Aboriginal communities on oral health and cardiovascular 
research since 2017.“”** The trust that members of Aboriginal communities developed for Poche 
Centre researchers was an enabling condition for the first stage of planning the precision medicine 


project. 


Building on this trust and previous work, the Centre has involved local Aboriginal people in co- 
designing a proposed precision medicine research project, including how the community will be 
involved as partners throughout the project. The Precision Medicine project aims to sequence 
participants’ DNA in order to create a clinically useful reference genome. This reference genome will 
contribute to improved health outcomes by improving the quality of precision medicine for 
Aboriginal people from communities in New South Wales. For example, it has the potential to 
improve prediction of cardiovascular disease and enable more targeted interventions for certain 


cancers and rare genetic conditions. 


Methods 


Case study method 


A reflective case study method is used here to describe my work. The method and structure of this 
reflective case study is informed by the ‘reflection cycle’, and other relevant reflective case 
studies.18342844 The stages of a reflective case study are (1) describing what happened; (2) reflective 
observation, including thoughts and feelings arising from what happened; (3) summarising relevant 
theory in order to explore what happened in the context of contemporary knowledge and evidence; 
(4) summarising any future action, and how this was informed by the previous stages. Data sources 
for this case study include a reflexive diary written throughout the research process, emails, meeting 


notes and relevant documents associated with the project. While methodology for data analysis for 
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a reflective case study is not prescriptive, in order to improve validity, a colleague from the Poche 


Centre (KG) was involved in checking themes, reflections and learning points from the process. 
Paradigms 


The scoping review | conducted of international genomic research identified community-based 
participatory action research as the method with the most potential in this respect.?” The work of 
the Poche Centre is informed by ‘collective impact’ method of co-design, which is a process for 
solving problems based on agreeing a common agenda, communication, “mutually reinforcing” 
activities, project support and involving multiple stakeholders in evaluation.*°°4°? 

In this thesis, participatory action research is used as an umbrella term which describes a number of 
related approaches, including forms of action research which embrace a participatory philosophy 
and include ‘co-design’, ‘collective impact’ and ‘co-production’ of research. °°") During the 
participatory process, Poche researchers ensured researchers and other relevant stakeholders 
(including community members) “work together, sharing power and responsibility from the start to 
the end of the project”,®° including knowledge generation and translation. ®°3°! The project will 
continue to use the paradigm of participatory action research, which attempts to reduce health 


inequalities by supporting people to be involved in research protocol design, ethical review, data 


collection, reflection and, ultimately, in action to improve their own health. 128 


It is important to note that the co-design process described here is a starting point for future 
research co-design and planning, and that only once careful and detailed co-design has taken place, 
are ethics applications and research protocols produced. This is in contrast to other kinds of research 
projects which gain ethics approval, publish protocols and then seek feedback through consultation 


with potential participants. 
Stages of research 


In 2018 | was invited to volunteer with Sydney University’s Poche Centre for Indigenous Health in 
order to support with their planned precision medicine project. | attended online meetings and 
commented on shared documents in order to support the co-design of the methodology; helped 
create learning materials and videos; made presentations to the Poche Centre Board about my 
research, including STARDIT; and supported with ethics applications. The stages of the co-design 


process are summarised in Figure 8.1. 
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Figure 8.1: Stages of research co-design with Aboriginal communities 


+ 2017 +e 2018 —————> 2019 ———————————> 2020 


Stage 2: Co-design 


2021 ————> 2022 — 


Pre-2017 2018 201$ 2020 2021 2022 
Poche Centre Relationship Study team Community Co-designing Ethics applications Protocol co-designed, Project planning continues, 
worked with development, established with gives consent, research protocol, @PProved, other published in 2021. including seeking research 

Aboriginal multiple meetings Aboriginal support given and ethics eate STARDIT embedded in funding 

communities on with Aboriginal representatives, by Aboriginal applications project design, prospective 
other research community research proposal Community STARDIT report published 
projects members shared Controlled 
Organisations 


At each stage of the process, stakeholders were involved in the following tasks: 


g igj OŁ Q gan ABE £8 Og STARDIT 


Checking 
Gr document 
application and wording 
protocol 


As part of the planning and co-design process, | was invited in 2019 to travel with staff members of 
the Poche Centre to meet members of a remote New South Wales community (see Figure 8.2, 
reused here with permission).*°* This meeting was one of a number of meetings with this community 
which took place before and after this particular one. The purpose of the meeting | attended was to 
meet with community representatives and members of the community to discuss any next steps 


regarding partnership and co-design. 


The meeting itself was a formal meeting, and thus followed respectful cultural protocols. It was 
attended by an Aboriginal elder who ‘loaned their trust’ to the Poche Centre researchers. The Elder 
welcomed the Poche Centre Director Boe Rambaldini, who formally responded, and then introduced 
members of the Poche Centre team, including Kylie Gwynne, who presented to the meeting 


attendees about the proposed project. 


My tasks at the meeting included talking with participants, making notes and sharing any 
recommendations with the Poche Centre staff, including assisting with writing a research protocol 
and integrating STARDIT into planned ways of working. After the event, in 2019, formal consent from 
the community was obtained via a community survey. In addition, letters of support were given 
from two Aboriginal Community Controlled Organisations. Learning from this event informed 
subsequent research planning, governance planning and associated ethics applications in 2019 and 


2020, and the published protocol in 2021. 


In parallel with this work, | was invited by the staff from The National Centre for Indigenous 


Genomics to be a co-author on a peer reviewed article, along with a number of Poche Centre staff 


230 


connected with this project. The article was published in 2020 and is titled ‘Equitable Expanded 


Carrier Screening Needs Indigenous Clinical and Population Genomic Data’.° 


Figure 8.2: Precision Medicine project meeting participants 2019 


pyas 


Photo published by the Poche Centre for Indigenous Health, 2019, showing Precision Medicine project meeting participants. 452 


Results 


Results from the participatory action research process included learning from the co-design process 
which informed the protocol creation, and the co-creation of the protocol itself. In parallel to this 
participatory process, work with the National Centre for Indigenous Genomics (NCIG) on an article is 
also summarised, as discussions with staff from the NCIG informed Poche’s participatory action 


research process, and learning from this process also informed the article led by the NCIG. 
Learning from the co-design process 


Working with staff from the Poche Centre to co-design participant information provided an 
opportunity to explore the attitudes and values of both the Poche Centre staff, and members of the 
communities they hoped to involve. While my initial work was limited to online meetings and 
commenting on shared documents, it provided an insight into linguistic expectations, agreed 


terminology and the values which are associated with that. For example, discussions on the usage of 
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the word ‘race’, and discussions about appropriate terminology with regard to the use of terms such 


as ‘Indigenous’, ‘Aboriginal’ or ‘community member’. 


The in-person co-design process helped elucidate concerns regarding the proposed genomic 
research. Some members of the community demonstrated a good understanding of how genomic 
variations can contribute to risk of disease, and how these can be inherited. While community 
members appeared supportive, a number of concerns were raised during the co-design process (by 
either Poche Centre staff or community members). These concerns included genetic discrimination 
in employment, difficulty in obtaining insurance, and inappropriate use of genomic data in law 
enforcement. It was therefore necessary for the co-design process to balance these concerns with 


the researchers’ requirements with regards to combining data. 
Co-created study protocol 


While subsequent planned work was delayed and disrupted by the COVID-19 pandemic, in 2021 the 
study protocol ‘A Pathway to Precision Medicine for Aboriginal Australians: A Study Protocol’ was 
published.? It is the result of the co-design process conducted by the Poche Centre with a number of 
Aboriginal communities and other relevant experts. The protocol outlines the background to the 


study, planned methods and phases and discusses the implications of the study protocol. 


By working closely with the Poche Centre staff, | was able to ensure that while | was myself learning, 
knowledge and experience gained from my PhD were also able to inform the study protocol. This 
included results from the scoping review, learning from STARDIT and other relevant professional 
experience informing my feedback. | worked on multiple versions of the research protocol, giving 
feedback, writing content and editing final versions. | was able to explain STARDIT to Poche Centre 
colleagues (including via invited presentations to the Board), and was able to integrate plans to use 
STARDIT in the study protocol. The study protocol is summarised in ‘Table 8.1: Study protocol 
summary’, which is adapted from content from the peer-reviewed protocol and the article | wrote 


for the Journal of the Consumers Health Forum of Australia.2? 


Table 8.1: Study protocol summary 


Background Genomic precision medicine (PM) utilises people’s genomic data to inform the 
delivery of preventive and therapeutic health care. PM has not been well- 
established for use with people of Aboriginal and Torres Strait Islander ancestry 


due to the paucity of genomic data from these communities. We report the 


232 


development of a new protocol using co-design methods to enhance the 


potential use of PM for Aboriginal Australians. 


Methods 


This iterative qualitative study consists of five main phases: 


Phase-! will ensure appropriate governance of the project and establishment of a 
Project Advisory Committee, following an initial consultation with the Aboriginal 


community. 


Phase-Il will invite community members to participate in co-design workshops. 
Using a combination of face-to-face events and online tools, Aboriginal 
community members and the Armajun Aboriginal Community Controlled Health 
Service will be involved in co-designing every stage of the project, including the 


ways in which people are involved. 


Phase-lll Chief Investigators will participate in co-design workshops and 
document generated ideas. Phase-IV notes will be analysed thematically with 
Aboriginal community representatives, and the summary will be disseminated to 


the communities. 


Phase-V we will evaluate the co-design process and adapt our protocol for use in 
partnership with other communities. We propose the use of Standardised Data 
on Initiatives (STARDIT) to report, evaluate, and keep public records of every 
stage of the co-design process. Aboriginal community members will help 
evaluate the project, including how people were involved. This will help appraise 


effective ways of working and identify learning points and best practice. 


Discussion 


This study protocol represents a crucial first step to ensure that PM research is 
relevant and acceptable to Aboriginal Australians. Without fair access to PM, the 
gap in health outcome between Aboriginal and non-Aboriginal Australians will 


continue to widen 


Influencing articles by the National Centre for Indigenous Genomics 


The article ‘Equitable Expanded Carrier Screening Needs Indigenous Clinical and Population Genomic 


Data’ had over 40 co-authors, and accordingly, there were multiple perspectives which we 


attempted to incorporate, and not all feedback from co-authors could be integrated. While my tasks 
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were limited to commenting on the multiple versions shared with the co-authors, my feedback that 
concepts of culturally safe and community led involvement should be recommended was included, 


and featured as follows in the abstract: 


[Aboriginal and/or Torres Strait Islander Australians] require culturally safe, community-led 
research and community involvement embedded within national health and medical 
genomics programs to ensure that new knowledge is integrated into medicine and health 
services in ways that address the specific and articulated cultural and health needs of 


Indigenous people.” 


While my recommendation that STARDIT be proposed in the article as a way of supporting this work 
was not integrated, the subsequent adoption of STARDIT by the Australian Genomics project ‘Involve 
Australia’ shows the work has relevance, as the remit of that project includes involving Aboriginal 


peoples in genomics research. 


Discussion and reflections 


Exploring perceived risk and benefits 


When working with Indigenous peoples in countries such as Australia, it is important to understand 
and acknowledge both the history and continued evidence of systemic racism and structural 


violence towards Aboriginal peoples.*°?4%4 


Some issues are common concerns of Aboriginal people in genomics research, including sovereignty 
over samples, and benefits and risks of genomic research to communities.” *°” One concern shared 
by Aboriginal community members and study team members was the risk of DNA data being be used 
for law enforcement and criminal justice, rather than strictly research. Owing to Australian law, such 
use could not be ruled out by researchers, and was thus an important risk to consider when co- 
designing this protocol. Accordingly, the project was co-designed with samples which were de- 


identified, and data analysis methods which combined reference genomes from multiple individuals. 
Getting the language right 


In an early version of a participant information sheet which | was asked to proof-read, the second 
sentence stated, “DNA is different but a lot of our DNA is shared, especially by people of the same 


race”. While the word ‘race’ was used as a plain English short-hand, | challenged the use of it as, by 
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the very act of using the word, it implied the researchers, and thus the project itself, believed in and 
endorsed the concept of ‘race’. Concepts described by words such as ‘race’ and ‘ethnicity’ have 
contested boundaries.*** A word like ‘ethnicity’ can be an unhelpful grouping in public health, as it 
”functions like a wild-card which can readily be inserted into a string of related concepts”, blurring 
inherited characteristics with learned behaviour, often used as a euphemism for race.^? For 
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example, ethnicity can be defined as referring to ‘cultural traits’ exclusively”, while other 


definitions also state that ‘racial’ origins can define it. ^51 


Moreover, words like ‘ethnicity’ and ‘race’ have amorphous meanings with parameters that are 


”context-dependent”*”> 


, with the usage of terms like ‘race’, ‘ethnicity’ and ‘ancestry’ changing over 
time.*© The Nuffield Council on Bioethics points out (in a report about pharmacogenomics) that 
“Race and ethnicity cannot be given precise biological or genetic definitions”.** Similarly, other 
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authors agree that the idea of race is ”a social invention with no biological validity”“™ or a ”socially 


constructed” category." 


Targeting public health messages at specific populations defined by race “may reinforce tendencies 
to view race as a biologically defined phenomenon” .*® The Nuffield Council on Bioethics 
recommend being "sensitive to the potential for misunderstanding and prejudice arising from racial 
stereotyping”. “© Biological determinism and racial essentialism both posit that the concept of ‘race’ 
can be defined precisely.“ Once regarded as “an ugly component of a bygone era of social 
Darwinism and race-based eugenics research supported by racially exclusionary and discriminatory 
laws, overt policies, and hostile attitudes”, such modes of thinking have “invaded contemporary 


study of genetics and genomics”.*™% 


It should be carefully noted that while there is no agreed biological definition of ‘race’, racism (a 
result of a belief in ‘race’) clearly does exist, and has profound implications for people’s human 
rights, social and health outcomes around the world. For example, while there is a rejection of race 
concepts among professional anthropologists and a consensus that there are no human biological 
races, there is a recognition that race exists as lived social experiences that can have important 
effects on health. 85325) As a result, whether biologically definable, the social realities of people’s 
attitudes about race and ethnicity also influence attitudes about engagement or involvement in 


genomics research. 
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While contemporary language would talk of concepts of ‘ancestry’, the Australian constitution uses 
the word ‘race’ on a number of occasions and gives ”special legislative treatment” to people 
perceived as different races.*©° Other legislation from 2013 defines ‘Indigenous’ as ”a member of the 
Aboriginal race of Australia”.*°” Historically, in particular during the Stolen Generations era of 
Australian history, people used to use fractions to try to determine and describe an individual’s 
ancestry, and to what extent they are “descended from the original inhabitants of Australia” .*©° As 
recently as 2020 the Australian Minister for Indigenous Australians cited that there are people in 
society still "who live with the notion that we take still the old practice of full bloods, half cast, 
quarter cast, quadroon”,*© labels that were used in Australian legislation up to 1936.7 
Controversial campaigns by some political parties in 2019 have sought publicity by calling for a 
biological definition of race in order to label people as ‘Aboriginal’ according to an external objective 
measure, rather than the current system of self-identification, despite there being no agreed ethical 


foundation for doing this, let alone a method.*”°471 


However, the precision medicine projects seeking to ensure data is useful for people of Aboriginal 
and Torres Strait Islander ancestry, and other Indigenous peoples around the world, must navigate 
the complex historical and contemporary contexts of racism carefully. Involving potential 
participants and community members in every stage of an initiative means it is more likely to be 
acceptable, relevant and culturally safe. This includes involving people in checking any information 


resources that will be shared and assessing if planned data access protocols are acceptable. 


After discussion, the word ‘race’ was removed in the final version of the document sent out. For me, 
the incident reinforced the importance of not only involving potential stakeholders in co-designing 
resources and checking language, but also reporting transparently how people will be involved, were 


involved and any impacts. 


In order to help navigate the complex language surrounding biological ancestry and learned culture, 
STARDIT has been designed to align with models of self-identification. Empowering people to chose 
their own labels and groupings (rather than be passively labelled or grouped) can help ensure such 
research aligns with the values of potential participants and does not perpetuate racist constructs 
and labels. For example, when STARDIT is used to report how people have been involved in an 
initiative, people can choose to self-identify with labels of their own choice, including ‘Aboriginal’, 
‘Indigenous’ or ‘community member’, removing the requirement (and the associated power) of 


researchers to assign these terms to people. STARDIT can be used to both plan and report each 
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stage of any such involvement, providing transparency for potential participants, and helping ensure 


accountability to those protocols by the future research projects. 


Similarly, involving people in deciding research priorities in a transparent way can also ensure 
research aligns with perceived priorities. Deciding what gets researched and what does not can 
create inequality. For example, a paucity of data about Aboriginal peoples means that existing health 
inequalities might be amplified. Reasons for a lack of research in a certain field (contributing to 
ascertainment bias in genomic data) might be cultural, financial or geographic. Using systems like 
STARDIT can help map both priorities, linguistic terms, co-define meanings and the preferences and 
values of multiple stakeholders, allowing an evidence-informed approach to prioritising, planning, 


doing and evaluating future genomic research. 
Co-defining ethical research 


Ensuring research is conducted ethically requires an ongoing and transparent process to involve 
people in defining what ethical means, and ensuring research is conducted within those defined 
boundaries. While such ethical processes need oversight, they also need to ensure that they are 
also not inhibiting research from being done which could save and improve lives. The Australian 
Department of Heath’s 2021 consultation on ‘Aboriginal and Torres Strait Islander Genomics Health 
Guiding Principles’ is a demonstration of involving people in helping shape such ethical principles for 
future genomic research with Aboriginal peoples.*”? One principle states that “projects should be 
conducted through co-design processes”. “°? The published protocol demonstrates how such co- 
design methods can be reported, helping ensure future ethical oversight can be managed in 
partnership with Aboriginal people. STARDIT can be used to describe who was involved in co- 
designing every aspect of the research, and who was involved in making ethical decisions, providing 


more transparency to the entire process of ethical oversight. 
Strengths and limitations 


This case study was designed to describe the careful, complicated and often time-consuming work 
involved in following best-practice when working with Indigenous peoples to co-design genomics 
research. While a protocol for genomics research might not seem like a research output in itself in 
some disciplines, in the participatory action research paradigm, this protocol represents an 
important output in itself. It represents a free publicly accessible, peer-reviewed plan for research, 


co-created with a community, alongside different experts. The planned reporting of the case study 
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using STARDIT also ensure the project will continue to provide publicly accessible information about 


how people will be and have been involved. 


While this case study is not an appraisal of the protocol itself, it is worth noting that Aboriginal 
community members were not only involved in every stage of the co-design of the protocol, but 
according to the protocol, will remain involved throughout every stage of the planned research. 

The data sources for this case study were limited by ethical considerations. Other than the protocol 
itself, data was limited to emails, conversations and documents shared between the study team. 
Using any data generated by people from the communities involved, including using direct 
quotations, would require an entirely separate ethics process, and was never part of the plan for this 
case study. While this limits the depth of the data that this case study could access, it also provides 
an additional learning point about the complexity of ethics processes, and how planning any co- 
designed project needs to allow for both the workload and the time required for the ethical conduct 


and ethical approval of research. 
Conclusions 


The work described in this chapter provides a clear methodology for both planning and reporting co- 
design methods in genomic research with Aboriginal people. STARDIT can be used to create public 
records of every stage of the co-design process, including data about who was involved, how, and in 
which tasks. Information about the co-design method, the data analysis methods, reported enablers 
and barriers for involvement, and any other impacts and outcomes from the co-design process can 
also be reported. Such data improves transparency of terms such as ‘co-design’ and allows improved 


scrutiny and ethical oversight, including from potential or current research participants. 


STARDIT MICRO Report: A Pathway to precision 


medicine for Aboriginal Australians 


This STARDIT MICRO report contains the minimum data required for a Standardised Data on 
Initiatives Alpha version report (STARDIT Alpha).*°* This STARDIT MICRO report contains additional 
data relevant to the protocol ‘A pathway to precision medicine for Aboriginal Australians’. The full 
prospective STARDIT report for this protocol can be found in the Appendices under the heading 
‘Prospective STARDIT Report: A Pathway to precision medicine for Aboriginal Australians’, with a link 
to the online ‘living’ Beta version of the STARDIT report available in the references.*”° 
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Name STARDIT MICRO Report: A Pathway to precision medicine for Aboriginal Australians 


Involvement 


Who will be Group 1: Academic investigators. 


involved Group 2: Aboriginal community representatives. 


Group 3: Aboriginal community members involved in co-design and consultation activities. 


Specific tasks Group 1 and Group 2: Project design, ethics applications, planning and delivering co-design processes 


and analysing data from co-design activities. 


Group 3: Face-to-face and online events, consultation processes, and checking data analysis. 


Outcomes (of | Involving Aboriginal community representatives and members will ensure planned activities 


involvement) (including research methods) are appropriate, culturally safe and effective. 


Group 1: Academic investigators will be volunteering their time and may be 


named as authors in peer-reviewed publications. 


Group 2 and 3: Aboriginal community representatives and community members 


may be paid for their time. 


N/A 

Data 

Who is the data from this As part of the co-design process, data sharing (including returning data and results 

intervention shared with? to participants), governance (including how to plan the management and storage 
of sample and DNA data), and data sovereignty will be agreed with participants 
and affected Aboriginal communities. 
Data transference processes used in the project will be consistent with the 
principles of participatory action research, where stakeholders collectively decide 
upon roles, responsibilities and data access. 

How is it stored and hosted? As part of the co-design process, stakeholders will collectively decide this, along 
with appropriate biobanking methods. 

Who is analysing the data? Stakeholders will collectively decide this as part of the co-design process. 

How is information about this Social media, mass media communications, policy papers, conference 

data disseminated? presentations and open access journal publications, short video, podcasts and 
public events. 
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How is the data FAIR??° 


Stakeholders will collectively decide this as part of the co-design process. 


Impacts and outcomes 


What new knowledge has been 


generated? 


The project intends to: 
1. inform best practice in co-design with Aboriginal communities 


2. answer important questions about people’s preferences about data ethics, 


security and quality associated with genomic research 


3. map people’s preferences using the STARDIT-Preference Mapping tool. 


Outcomes — Describe how the 
learning or knowledge generated 
from this initiative has or will be 


used. 


Using a co-design process: 

1. promotes usage of health services, elicits superior health outcomes and saves 
time 

2. builds strong and committed community partners 


3. enhances skills and knowledge in the Aboriginal community about genomics 


and health. 
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Chapter 9 — Results: Comparison of all 
case studies 


Comparison of all case studies 


This chapter will summarise the results from all case studies and answer the questions: 
e What does a comparison of the data analysed from each project reveal? 
e What are the themes, generalisations, similarities and differences between these case studies? 


e What are the significant observations and learnings from these observations? 
It also includes a comparison of results from STARDIT reports from this doctoral research. 


As this chapter refers to participants from multiple case studies, when participants are quoted they 
will be identified using the following format: 

e Shared Ancestry - Participant 1 = [P1-SA] 

e  ausEE Participant 1 = [P2-ausEE] 


e ASPREE participant 1 = [P1-ASPREE] 


Quantitative cross-case analysis 


During this thesis, | developed a standardised way of planning, reporting and evaluating stakeholder 
involvement in order to improve future genomics research using standardised data (STARDIT), 
including mapping the preferences of different stakeholders using standardised preference mapping 


(STARDIT-PM). 
Baseline and follow-up preference mapping 


Below is a comparison of the quantitative data gathered from all case studies where both baseline 
and follow-up data was collected, in order to map changes in preferences about involvement in 
research. Data came from the question ‘Which aspects of any future research genomics research 
should be influenced by the following?’. Participants answered using a tick-box grid to indicate which 
aspects of research they felt should be influenced by which stakeholder groups. Figure 9.1 shows 


this grid. 
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Figure 9.1: Preference mapping grid 


Anyone who 


Everyone (any might be Only people Only people Only people 
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public whois affected bythe affected bythe participating in professional 
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Finding questions to ask 
(identifying research topics) n = = = = 


Deciding which questions to 
prioritize and fund Id ~ - C 


Deciding how to try and 
answer the question (the O O oO O B 
research method) 


Attempting to answer the 
question (carrying out the 
research, including A =- = 5 = 
collecting information) 


Trying to understand if it is 
possible to answer the 
question (analyzing the 7 i i ii ğ 
information) 


Sharing the information that 

has been found, and any 

answers that may have O O O O O 
emerged (dissemination and 

publication) 


Ensuring that any 

information or answers are 

able to be used to help = ts = 
people in practice, policy or a = = = 
future research (sometimes 

called research translation) 


Deciding if the way of asking 
the question and all the 
other stages of the research . 
were appropriate (evaluating jn = = = n 
the research method and 

any impacts) 


Designing how people are 
involved in the research j 5 z g z. 


In the ausEE study, of the 26 people who gave consent and completed the initial survey, 15 
participated in the online discussion and 12 completed the follow-up survey. From the 12 responses 
to the baseline and follow-up survey, participants gave 41 responses. In the ausEE study 59% of 
participants’ responses showed a change towards ‘widening’ their view of who should be involved in 
research to include more people (N=24/41), 34% stayed the same (N=14/41) and 7% narrowed 
(N=3/41). 
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In the Shared Ancestry study, from the six participants who completed both the baseline and follow- 
up surveys, a total of 54 responses were given. Of these, 35% showed a change towards ‘widening’ 
involvement (N=19/54), 8% ‘narrowed’ (N=8/54) and half stayed the same (N=27/54). Table 9.1 
below combines the results from the 18 participants across the two studies who completed both the 
baseline and follow-up surveys using identifiable data at both stages, providing a total of 95 
responses to the questions. 45% of participants’ responses changed to ‘wider’ (N=43/95), 43% 
stayed the same (N=41/95) and 12% narrowed (N=11/95). More detailed data can be found in the 


Appendices under ‘Detailed baseline and follow-up data on preferences for involvement’. 


Table 9.1: Summary of baseline and follow-up preferences 


Who should influence which aspects of Change to 
Change to wider No change 

research? narrower 

ausEE sub-totals 24 14 3 


(total of 41 responses) 


Shared Ancestry sub-totals 19 27 8 


(total of 54 responses) 


Standardised Preference Mapping (550) 


This analysis combines all public domain data from case studies using the preference mapping tool 
STARDIT-PM to categorise the data into certain areas’, allowing a comparison of data from 83 
different stakeholders from the three case studies where preferences were mapped, including the 
facilitators of the online discussions (ausEE and Shared Ancestry) and the study team members from 


the ASPREE case study. 


Preferences were recorded from all data sources, including the initial surveys, online discussions 
with participants, online facilitator discussions, follow-up surveys with participants, surveys with 
facilitators and study team members and emails from the ASPREE study team. If the same person 


made the same point at different stages, this was counted as one view. 


Questions in online discussions and surveys were designed to provide data for these categories. This 


parity in data collection and analysis has allowed this data to be combined and compared from all 
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the different stakeholder groups, thus providing more statistically useful data and providing a proof 


of concept for using this method of data collection and analysis. Table 9.2 provides a summary of the 


STARDIT-PM data, with more detailed data in the Appendices ‘Quantitative data from STARDIT-PM 


cross-case analysis’. 


Table 9.2: Summary of STARDIT-PM cross-case data 


Shared Online facilitators 
ASPREE | ausEE ancestry (ausEE and Shared 
Data set totals participants | participants | Ancestry) 
Total number in dataset 42 26 12 3 
Views on who should be involved | 31 17 10 0 
Views on who should do which 
tasks 26 10 8 0 
Views on modes of 
communication 32 3 4 0 
Views on what methods should 
be used 25 6 5 0 
Views on enablers of involvement | 25 10 7 3 
Views on barriers of involvement | 10 5 6 3 
Views on what the outcome or 
output of the research or 
involvement in research could be | 11 3 2 0 
Views on which stage of the 
research people should be 
involved 24 0 1 0 
Research data 19 0 3 0 
Views on how learning from this 
research could be used 0 0 8 0 


Most stakeholders expressed views on who should be involved (70%, n=58/83). Views on enablers of 


involvement was the second highest area people expressed views in, with all three facilitators 


sharing views on this. Views on who should do which tasks were shared by 44 people, with all case 


studies having a relatively high number of responses for this category. This standardised data 
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provides useful information about different stakeholders’ preferences that could be useful in 


informing the design of involvement in future research. 


Analysis using STARDIT-PM allows a quantitative comparison of which stakeholder groups shared 
views about which areas, allowing a better understanding of which groupings those preferences are 
from. When this kind of data is structured using STARDIT, it allows more complex statistical analysis, 
including giving ‘weight’ certain groups’ preferences. For example, if more researchers than 
participants have shared perspectives, the researchers’ preferences can be grouped and compared 


to the preferences of the participants. 


Learning from such statistical analysis might suggest areas where people need learning and 
development interventions in order to support informed decision making. For example, if most 
research participants have not shared preferences about data sharing and who should access their 
genomic data, it might mean they don’t understand the principles or concepts, not that they don’t 
have preferences about who can access this data. This might indicate an area for further learning 
and development in order to ensure their views and preferences are able to influence future 


genomic research. 


Raw data about how participants from the ausEE and Shared Ancestry case studies answered both 


the baseline and follow-up questions can be found in the Appendix in the section ‘Cross-case 


analysis data from ausEE and Shared Ancestry case study’. 


Standardised Data on Initiatives (STARDIT) data summary 


This section compares the number of different methods and tasks of involvement, and the reported 


impacts. All studies reported impacts from the process of involving people. 


Table 9.3 summarises the quantitative data from the three case studies where it was reported 
(ausEE, Shared Ancestry, ASPREE), noting the Indigenous Precision Medicine case study is a 
prospective report. A more detailed version of Table 9.3 is included in the Appendices in the section 


‘ 


Detailed quantitative analysis of STARDIT data categories’. 
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Table 9.3: Standardised Data on Initiatives (STARDIT) quantitative summary 


Study Number of methods Number of reported Number of Total number of 
[STARDIT Data Category communication modes | reported tasks impacts 
‘Method of doing task?'] [STARDIT Data [STARDIT Data [STARDIT Data 
Category Category Category 
‘Communication 'Tasks'] ‘Involvement 
modes] outcomes, impacts, 


learning or outputs'] 


ASPREE | 4 3 4 7 
ausEE 3 3 4 6 
Shared | 3 3 5 8 
Ancestry 


While the reductive quantitative analysis in Table 9.3 can be useful for large datasets (such as the 


27) it has been included here to indicate the 


2019 review of global genomics research projects 
limitations of such an approach with a small dataset. Noting the standardised data can be added to 
any future datasets, while the use of this quantitative data is limited, it could be combined in the 
future with other standardised data to gain more insights. For example, the charity Science for All 
used the STARDIT-PM tool to map preferences of the public about who should be involved in 
research and reported it,*”? using a licence which allows this data to be compared and combined 
with the data in this thesis, and future datasets in order to provider richer analysis (see Appendix 


‘ 


STARDIT-PM preference mapping data from public survey for comparison’). 


The next section includes a qualitative thematic analysis from the standardised data, which provides 


more insights into the quantitative data above. 
Qualitative cross-case thematic analysis 


This section summarises the results of a thematic analysis of all case studies. It is divided into 


sections which explore themes from: 
1. the two online discussion case studies (ausEE and Shared Ancestry) 
2. the ASPREE, ausEE and Shared Ancestry case studies 


3. all case studies (ASPREE, ausEE, Shared Ancestry and the Indigenous Precision Medicine 


project). 
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First, (1) the ausEE and Shared Ancestry case studies are compared, as the methods were very 
similar. Secondly, (2) the three case studies where there was standardised data collection (including 
STARDIT-PM) are compared, and thirdly (3) all case studies are compared, including the Aboriginal 
Precision Medicine project, which was a reflective case study and where STARDIT reporting is used 


prospectively. 
1. Themes from the two online discussion case studies 


Online discussions for both the ausEE and Shared Ancestry studies demonstrated a high level of 
genomic literacy and understanding of ethical issues among participants. In each discussion, at least 
one person stated that they had been involved in genomics research as an investigator, not as a 
participant. Participants reported learning more about genomics as a result of taking part, and 
indicated that the learning resources provided were helpful, particularly, the visual summary of 


‘Public involvement in global genomics research: A scoping review’ (see Figure 4.2 in Chapter 4).*74 


Participants shared views about the limitations of one-to-one ‘one way’ interviews. One person 
commented that they can be “quite straight jacketed with circumscribed questions” [P4-SA]. Among 
their advantages, online discussions permitted participants to ask questions and enabled researchers 


to gather data they might not otherwise have planned to collect. 


Some participants reported that the pace of discussion was too fast and “moved on too quickly” [P7- 
SA] [P4-SA] [P5-SA], which risked leaving “some people behind” [P3-ausEE]. However, owing to the 
asynchronous design of the study, participants were able to contribute to discussions and catch up in 


a flexible way, and this was reported as an important enabler of participation in both studies. 


In both case studies, participants reported a desire to help researchers. Notwithstanding, in both 
discussions, concerns were raised about researchers’ intentions regarding genomics research 
(generally, not specific to these studies). Participants preferred transparency about this and were 
concerned about “hidden motives” [P2-SA]. The importance of determining the “purpose of 
research” was raised in both discussions, with participants suggesting that they should be involved in 
“agreeing purpose” [P7-SA]. Learning from this process influenced the design of the STARDIT Beta 


version (0.2), in which the purpose of research was added as a data field. 


In both studies, participants discussed the concept of everyone being involved, but noted that some 
people should be more influential than others. In the Shared Ancestry study, one participant 


identified medical scientists, social scientists, psychologists and the general public as groups which 
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should influence research, but noted that not all “groups should have equal influence” [P10-SA]. One 
participant from the ausEE study stated that people with a rare disease and their families are “likely 
to have different priorities from scientists” [P16-ausEE]. A Shared Ancestry participant concluded 
that “we're all biased; whoever is affected by a condition is likely to want it prioritised” [P12-SA]. 
Similarly, a participant from the ausEE study stated that people affected by Eosinophilic 
Gastrointestinal Disorder “certainly do” have a vested interest in genomics research into that 
condition [P3-ausEE]. These discussions highlighted the importance of being able to use STARDIT to 


map and report on the different interests and perspectives of diverse stakeholder groups. 


One notable difference between participants in the two case studies was that participants in the 
Shared Ancestry study appeared to find using the online discussion platform more difficult than 
those from the ausEE study. The latter reported no issues with the online discussion platform. As 
these participants were recruited from an existing online community, it is probable that they were 
more accustomed to that style of discussion and more familiar with using online discussion tools. 
This variation in reported difficulty highlights the importance of mapping the preferences of people 
who will be involved in research and this can be done using the preference mapping tool, STARDIT- 


PM, developed as part of the STARDIT framework. 


2. Themes from ASPREE, ausEE and Shared Ancestry 


Participant uncertainty about what they can offer 


All three case studies indicated that participants were willing to get involved and help improve 
research, but were uncertain about how they could get involved and, in particular, what tasks they 
could do. In the Shared Ancestry study, one participant stated that compared to “highly-qualified 
scientists who know what's possible” some “relatively ignorant lay people” will be limited in what 
they can offer at some stages [P7-SA]. Similar views were expressed by participants in the ASPREE 


study, in which participants were willing to help but were unsure as to how they could. 


Forty-five per cent of ASPREE participants understood the concept of involvement, but expressed 
self-doubt about how they could be involved (45 per cent, 9/20). While participants were happy to 
be involved, this was linked with phrases such as “I don’t know I’d have much to contribute” [P10- 


ASPREE]. 
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Similarly, among ASPREE participants who understood the concept of involvement and were willing 
to be involved, 25 per cent were unclear about what tasks they could be involved in and whether 
they would have sufficient expertise (25 per cent, 5/20). One participant stated that “my expertise 


wouldn’t be sufficient” [P2-ASPREE]. 


Participants under-valuing or not understanding what they can offer in regard to involvement 
suggests that learning and development may be a crucial intervention. Interventions that help 
people understand what involvement is and the tasks they can be involved in may help people value 


the perspectives they can offer and help improve research.” 
Desire to help and to know they have helped (146) 


One participant from the Shared Ancestry study wanted to know that their involvement had been 
“useful to the researchers” [P11-SA], while another stated that an outcome of being involved would 
be the “satisfaction of knowing that | may have contributed” [P10-SA]. This view was echoed in both 
the ASPREE and ausEE studies, where participants shared a desire to be involved in order to help 


both the research process and the researchers involved. 


It is interesting to note that this desire to help was not considered by study team members from the 
ASPREE case study. In the follow-up survey, one team member reflected that they had “worried too 
much” about the time burden of involving participants [MS]. This important finding shows the 
importance of mapping people’s preferences about involvement in genomic research, so that 


potentially incorrect assumptions about people’s willingness to be involved can be avoided. 
3. Themes from all case studies 


Impacts from involving stakeholders 


Involving research participants and study team members in all aspects of the research process 
yielded positive impacts in every case study (see Table 9.4 below). As the Indigenous Precision 


Medicine case study is a prospective protocol, it is not been included in this table. 
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Table 9.4: Summary of impacts from all case studies 


documents, meetings, 
interviews, group 


discussion. 


Number of methods: 4 


meeting, telephone 
interviews, face-to- 
face group 


discussion. 


Number of 
communication 


modes: 3 


documents (including 
participant 
information), 
commenting on 
research design, 
sharing views and 
perspectives, 


analysing data. 


Number of tasks: 4 


1. 


2. 


3. 


4. 


Study Reported methods of Reported modes Reported tasks Involvement outcomes, impacts, learning or outputs 
involving people [STARDIT Data [STARDIT Data [STARDIT Data Category 'Involvement outcomes, impacts, learning or 
[STARDIT Data Category Category Category 'Tasks'] outputs'] 
‘Method of doing task?'] | ‘Communication 
modes’ ] 
ASPREE Commenting on Face-to-face Reviewing Seven impacts on proposed study design from involving people: 


Recruitment and sample collection plan changed. 
Participant communication improved. 

Participants were involved in governance. 
Participants were involved in controlling data access. 


Participants were included on study advisory groups (including ethical 


oversight) using multiple communication modes. 
Feedback was provided to participants about the research. 


Learning and development opportunities were created for potential 


participants. 
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Study Reported methods of Reported modes Reported tasks Involvement outcomes, impacts, learning or outputs 
involving people [STARDIT Data [STARDIT Data [STARDIT Data Category 'Involvement outcomes, impacts, learning or 
[STARDIT Data Category Category Category 'Tasks'] outputs'] 
‘Method of doing task?'] | ‘Communication 
modes’] 
ausEE Commenting on Online shared Reviewing Six impacts reported from involving people: 


documents, survey, 


group discussion. 


Number of methods: 3 


documents, online 
survey, online text- 
based asynchronous 


discussion. 


Number of 
communication 


modes: 3 


documents (including 
participant 
information), 
commenting on 
research design, 
sharing views and 
perspectives, 


analysing data. 


Number of tasks: 4 


1. 


2 


3. 


4. 


Participants reported that learning resources were useful. 
Participants reported changed views as a result of participating. 
Participants reported enjoying the online discussions. 

Online discussions to be used in future research prioritisation. 
Participants asked to stay involved in the research. 


Participants reported improved understanding of how to get involved 


in research. 
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Study Reported methods of Reported modes Reported tasks Involvement outcomes, impacts, learning or outputs 
involving people [STARDIT Data [STARDIT Data [STARDIT Data Category 'Involvement outcomes, impacts, learning or 
[STARDIT Data Category Category Category 'Tasks'] outputs'] 
‘Method of doing task?'] | ‘Communication 
modes’] 
Shared Commenting on Online shared Reviewing Eight impacts reported from involving people: 
Ancestry | documents, survey, documents, online | documents 1. Improved understanding of genomics informed siblings’ 
group discussion. survey, online text- | (including participation in future research. 
PERE participani 2. Participants reported that learning resources were useful. 
Number of methods: 3 | asynchronous information), ma ; eee 
3. Participants reported changed views as a result of participating. 
discussion. commenting on 
; 4. Participants asked to stay involved in the research. 
research design, 
Number of sharing views and 5. Participants enjoyed the online discussions. 
communication perspectives, 6. Improved understanding of how to get involved in research. 
modes: 3 analysing data, 7. Co-design changed study design. 
contributing to 8. Method for future research co-design established. 


STARDIT report 


data. 


Number of tasks: 5 
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Summary of impacts 


This section provides a summary of themes from the impacts of involvement across the completed 
three case studies. Follow-up surveys were conducted with participants in the ausEE and Shared 
Ancestry case studies. Due to ethical restrictions, only ASPREE study team members were surveyed, 


so there may be multiple unreported impacts from this research. 


Potential participants in all case studies were involved in giving feedback on the planned process for 
involving people. Study team members from all case studies reported feeling that this improved the 


design of the respective studies. 


Participants from all studies asked to stay involved in the case studies, and participants from each 
were involved in producing peer-reviewed articles about the study — either as an author or by giving 
feedback on the manuscript and being named in the acknowledgements. In all cases, some 
participants from each study agreed to be named in the respective STARDIT reports and were 


coupled with the specific tasks they undertook. 


Participants from both the ausEE and Shared Ancestry case studies reported that the learning 
resources provided were useful, and helped improve their understanding of both genomics research 


and how people can be involved. 


Stakeholders from all case studies reported changing their views about involving people, either as a 
result of participating or being a study team member. All changed views represented a move 


towards valuing the importance of involving people in genomics research. 


Most responses from participants in the ausEE and Shared Ancestry case studies showed a 
‘widening’ towards a preference for more people being involved in genomics research. This 


preference and follow-up data were not collected for the ASPREE study. 


Participants from all case studies reported enjoying being involved and being part of the research 
process. With regard to preferences for involvement, some participants from the Shared Ancestry 
and ausEE case studies highlighted their preference for online discussions, reporting that these are 
more flexible and accessible and, thus, more inclusive. The Chief Executive of the charity ausEE 


reported she would like to use online discussions as part of future research prioritisation processes. 
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Participants from both the ASPREE study and the Shared Ancestry study reported they would like to 
be involved in future data access decisions. In the ASPREE case study this was incorporated into the 
design of the prospective multi-generational research study (MGRS). In the Shared Ancestry case 
study, participants reported that learning and changed views from participating in the online 
discussions had influenced their decisions both about invitations to participate in genomics research, 


and about how their siblings would like to be involved in future research. 
Different interests 


Participants from all case studies recognised different “interested parties” with regard to who is 
using data from genomics research and how [P7-SA]. According to participants from the Shared 
Ancestry study [P9-SA], private companies charging or benefitting from data was “distasteful”. This 
sentiment was echoed by a vote among ASPREE case study participants, all of whom stated that, 
while they were comfortable with their data being held by academics, they were not comfortable 
with data being held by a commercial company. However, one participant suggested not “ruling 
private companies out completely” from research [P9-ASPREE]. The ASPREE participants also 
questioned different interests in research, with one participant stating “I don’t trust people or 
organisations easily” and adding that “there’s vested interests out there — why are they doing it — 


what is in it for them?” 


In the ausEE case study, the themes of the ’intention’ and ‘purpose’ of research emerged, with a 
related discussion on vested interests. Clarifying the purpose or ‘intention’ “sets all direction and 
intention” (sic) of research [P3-ausEE]. The purpose or ‘intention’ of research should be transparent, 
with one participant asking if it is needs’ based or ‘reward’ based” [P3-ausEE]. AusEE participants 


also noted that they themselves have a vested interest in research [P3-ausEE]. 


One Shared Ancestry participant asked, “...there will be many interested groups, so which ones will 
be listened to?” [P4-SA]. One participant stated that “people who are not looking for personal gain, 
but who have a desire to improve quality of life and help us understand ourselves” should influence 
research [P4-SA]. A mistrust of for-profit research and “sponsored facts” [P5-SA] was reported by a 
Shared Ancestry participant. This sentiment was echoed across the other case studies. Concerns 
about data being used for criminal or politically motivated purposes was a significant consideration 


in the Indigenous Precision Medicine case study. 


Participants in the ausEE case study identified that the different interests of stakeholder groups 


affect not only research, but also health technology assessment processes. One ausEE participant 


254 


noted that rare conditions often do not get access to specialists or funded medicines. They identified 
that this might be because “we will never have the 'numbers' [for sufficiently powered trials]” as 
such trials are not possible with smaller groups of people [P3-ausEE]. Their view that the current 
evidence thresholds used by some health technology assessment processes may unfairly 


discriminate against rare diseases aligns with other studies.*”>*7° 


Participants also shared a mistrust of some organisations that are physically hosting data. The 
ASPREE participants also noted mistrust in for-profit social media companies, with one participant 
stating “I don’t trust any of the social media” [P17-ASPREE]. The decision to use Loomio, which is 
hosted on Australian Government servers, in the two online discussions for this research was 
vindicated by these comments and informed co-design of the research at an early stage. Attitudes to 


data sharing are explored further in the next section on ‘Data sharing’. 


STARDIT can be used to report different interests, including financial interests or personal interests, 
and was used to report on stakeholder concerns in the three completed case studies. Mapping and 
reporting of data in this way can improve transparency and accountability in both genomics research 


and health technology assessment (HTA) processes. 


Findings from the case studies reinforced the findings from the initial scoping review. Aspects of 
genomics research that are translated into clinical use can be described as genomic medicine 
(including genetic testing), which is then classified as ‘health technology’. Such translation of 
genomics research is affected by decisions about resource allocation.*””*”8 In the scoping review, 
reports of initiatives involving people in managing real or perceived conflicts of interest had 
significant implications for health technology assessment (HTA) policy around the world. For 
example, there is evidence that some industry-funded studies are more often favourable to the 
funder’s products than are non-industry funded studies.” This problem is compounded when 
those involved in the HTA decision-making process have access only to evidence which is itself the 
result of research agendas and which are not always aligned with the priorities of the people who 
the HTA process is intended to benefit (such as ‘payers’ or members of the public).3”231430 The 
STARDIT system provides a way to report on how different stakeholders have been involved at every 
stage of genomics research, including in prioritising research questions and scrutinising the HTA 
process. Reports from STARDIT are potentially important in maintaining public trust that HTA 


processes represent the public interest.?2°** 
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Data sharing 


Data sharing in relation to genomics research is a complex and ever-developing area. This section is 
divided into themes identified from the case studies which relate to data sharing. All four case 
studies reported discussions about data sharing practices, with the most in-depth discussion taking 


place between people affected by a rare disease in the ausEE case study. 
Poor data sharing practices negatively impact healthcare and research (100) 


Two ausEE participants expressed frustration at current data sharing practices, with one stating “| 
get so tired of 'fighting' for our corner” [P3-ausEE]. Frustration was caused at multiple levels 
including at the individual care level, healthcare system level (including health insurance), and 
international research level. One ausEE participant highlighted the fact that poor data sharing 
practices in routine healthcare negatively impact both care and research, including losing 
opportunities to identify potential research priorities [P3-ausEE]. “Mutual respect” was identified as 
an enabler of improved practices, for example, listening to the concerns of affected people and 


recognising their expertise [P3-ausEE]. 
Control of information 


The issue of who controls information, and to what end, was discussed by participants in a number 
of contexts, including in relation to open “borderless” access to information and the problems of 
paywalls [P3-ausEE]; “vested” interests in controlling information and technology — with people 
affected by a disease included in those with a vested interest [P15-ausEE]; the lack of “safeguards” in 
place which makes people feel unsure about sharing data on electronic health records, and how 


having genetic tests might affect eligibility for insurance [P15-ausEE]. 


Similarly, Shared Ancestry participants gave their views about with whom data should be shared. 
One participant expressed explicit concerns about sharing data for political or financial uses [P5-SA], 
with other participants agreeing. One participant stated “Research is for humankind. Its benefits 
should be available to all. Information should be for the most part easily available. If it's publicly 
funded, it must be publicly available” [P9-SA]. Cultural conventions around ownership of knowledge 
such as “patenting” were also challenged as forms of knowledge control which were viewed as 


|’ 


“unethical” in some contexts [P9-SA]. 


Access to information, including “borderless” access, was raised as an issue affecting who can be 
involved in research [P3-ausEE]. Participants reported that paywalls for articles have direct negative 


impacts on patients, carers and parents wishing to make informed decisions and “judicious choices” 
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about healthcare or wishing to get involved in research [P3-ausEE]. People affected by diseases exist 
“across economic and political boundaries” and access to medication is affected by “who owns, 
profits, controls, patents” [P3-ausEE]. Open data needs to be balanced against individual privacy 
concerns and safeguards are needed to protect against misuse of data. One participant added that 
knowledge should be shared openly, “... open access data makes real sense in terms of the most 
efficient way to find ‘like’ patients and bring us ever closer to treatments and perhaps a cure.” 
Participants from the ausEE study also shared the view that involving people affected by a disease in 


data sharing can “prompt” new areas of research [P3-ausEE]. 
Concerns about data privacy and misuse (643) 


Four ausEE participants mentioned insurance as an area of concern requiring public involvement and 
scrutiny with regards to data sharing. One person stated that it is a “valid community fear” that 
research data will be used “for” insurance companies or other for-profit organisations [P3-ausEE]. 
For example, one participant raised a specific concern around health research being used by 
insurance companies or other organisations “to be able to set new prices or exclusions” [P3-ausEE]. 
Involving insurance industry stakeholders was highlighted as a strategy to bring about greater 
transparency on the risks of participation in genomics research, so that people “don’t need to fear” 
participating in case it affects their insurance policies [P3-ausEE]. One participant stated that 
“security of information needs to be well protected” so it is not “used against an individual or group 
of people” [P3-ausEE]. Two other participants stated that they would not use electronic health 
records or genetic testing for their children “until the safeguards and the benefits are clearer” [P16- 
ausEE]. One participant stated that “progressive watering down of privacy protections” meant they 
were concerned their children would be “refused insurance because of a decision | have made” 


[P16-ausEE]. Other participants felt the same way about this [P14-ausEE][P25-ausEE]. 


A similar mistrust was echoed in ASPREE participants’ views about private companies and social 
media organisations. Seven ASPREE participants said they would be interested in being involved in 
decisions about data access, with all participants stating they while they were comfortable with their 
data being held by academics, they were not happy with them being held by a for-profit company. 
As previously mentioned, however, one participant said not to “rule private companies out 


completely”. 


Five Shared Ancestry participants raised a number of concerns about genomics research being 
misused for political purposes and “wicked ends” [P5-SA]. One participant stated that research 


should not be used “for political purposes” but indicated that they did not believe this was easily 
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prevented [P5-SA]. Another participant shared a view about “realism about the likely corrupt use of 
genetic information for political and financial purposes” [P2-SA]. A number of participants cited 
historical examples of genetic discrimination by regimes that had provided important learning for 
future genomics research. One participant cited the well-documented case of the large technology 
company IBM being complicit in enabling regimes to carry out forced negative eugenics policies.“ 
For this person, IBM were “the enablers for the Nazis’ ability to hunt down Jews and other 


‘undesirables’” [P6-SA]. 


The Indigenous Precision Medicine case study involved co-creating a research protocol. It required 
much careful discussion about ensuring that Indigenous community members were equal partners in 
all decisions about what kind of data was collected and how the data would be collected, shared and 
analysed. Concerns regarding genetic research — such as genetic discrimination in employment, 
difficulty in obtaining insurance, and inappropriate use of genomic data in law enforcement — all 
needed to be considered. It was therefore necessary for the co-design process to balance these 


concerns with the researchers’ requirements with regards to combining data. 


With the convergence of for-profit companies (such as social media companies and direct-to- 
consumer DNA testing services) storing and selling information about people, governments around 
the world are increasingly partnering with such companies to provide health services. It is therefore 
crucial to be able to map people’s preferences for how their genomic data (and other sensitive or 
personal data) is shared and with whom. As the STARDIT system offers a way to map and report on 
different stakeholders’ preferences with regard to data usage, it provides a practical and scalable 
solution to this complex issue. The STARDIT system uses both the ‘FAIR’? and ‘CARE Principles for 
Indigenous Data Governance’ to structure the data,” making it a particularly useful tool for 
genomics research involving Indigenous participants. Reports from STARDIT could be used by people 
to help them make informed decisions about sharing their own data or participating in genomic 


research. 
Different tasks for different people (322) 


Participants from all case studies recognised that it might be more appropriate to involve certain 
stakeholders in some tasks, rather than others, with involvement in the design-stage tasks flagged as 
being more practical than in later stages of the study. A Shared Ancestry participant asked “... can we 
be both subjects and supervisors — at the beginning yes, later, maybe not?” [P7-SA]. The ASPREE 


study participants shared the view that involvement at the design stage is more practicable. 


258 


The ausEE participants liked the idea of a multi-stage approach to involving people, using a 
combination of face-to-face and online discussions [P25-ausEE], and this aligns with other models 


used in research priority setting.43°? 


One Shared Ancestry participant commented that people (the public and research participants) will 
have a “variety of professional and technical and creative skills” which will be useful, with the most 
useful being “knowing ourselves” [P7-SA]. This mirrored comments in the ASPREE case study where 
participants noted that certain people have relevant skills and experience, and that “different 


personalities” might be more appropriately involved in certain tasks [P15-ASPREE]. 


The Indigenous Precision Medicine project has identified distinct tasks for different stakeholders, 
with members of the community involved in reviewing information and being part of consultations. 
According to the protocol, an independent Project Advisory Committee (PAC) will be established. It 
will be jointly chaired by an Aboriginal Elder nominated by the community and the Director of the 
Poche Centre for Indigenous Health of the University of Sydney, who is an Aboriginal Elder and 
senior researcher. The protocol also outlines how the training of local Aboriginal people as genetic 
counsellors and pathology collectors will be discussed with the local community. This variation in 
tasks for different stakeholder groups is a best practice example of power being shared 


transparently and different stakeholders having clear tasks and accountabilities. 
Involve people in designing involvement (117) 


Planning how people will be involved in every stage of a project is a complex process and requires 
input from the different stakeholders who will be involved. Findings from the review of genomics 
projects indicated that involving people in designing involvement would improve the effectiveness of 
such processes.’ The results from all case studies supported this, with involving people in designing 
the involvement process resulting ina number of positive impacts. These impacts are outlined in 


detail in each case study. 


One Shared Ancestry participant added that the public should have a voice in how “science and 
research can better involve” people [P5-SA]. This perspective aligns with findings from both the 
narrative review and the scoping review of public involvement in genomics. STARDIT was developed 
in response to findings and recommendations from the review of genomics projects.” Using 
STARDIT-PM for standardised preference mapping was a helpful tool to map people’s preferences 


within a standard framework. This led to development of guidance for planning and evaluating 
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initiatives using STARDIT which, in turn, informed the Indigenous Precision Medicine project 


protocol. 
Choosing what to know about your genome 


Participants in the Shared Ancestry study held the view that there might be important variations in 
the knowledge that people choose to have about themselves and their genomes. Similarly, 
participants from the ASPREE study said they would like to be involved in deciding how information 
is shared, as they had concerns about people learning things they did not want to learn. One ASPREE 
study participant considered that knowledge about whether someone was carrying a variation which 
pre-disposed them to Huntington’s could be a “poisoned chalice” [P4-ASPREE]. Such considerations 


were central to the protocol design of the Indigenous Precision Medicine case study. 


The finding that participants wished to be involved in deciding who decides, and who knows what, 
aligns with a 2015 United Nations report which poses questions of autonomy in genetics. Decisions 
must take into account both the person being tested and implications for relatives and communities 
of people who may share the same or similar genetic variations.>> While there are no simple answers 
to these questions, involving people affected in the decision-making process is essential. In February 
2021, for example, after careful consultation, results were returned to some ASPREE-XT 
participants.*®? The lead investigator (PL) for the ASPREE project said that data from the ASPREE case 
study about people’s preferences about access to their own genomic data and analysis was used 


when making the case to ethics boards for returning data. 


The STARDIT system allows standardised reporting of who has been involved (including people with 
specific genetic variations) using both FAIR and CARE criteria. It provides transparent data about the 
different preferences of people with variations. It helps inform decision making and allows others to 


critically appraise how different stakeholders have been involved. 


Ivory towers, silos and bubbles: labelling the academic constructs and mapping 


ethical grey areas 


Research for this thesis used the paradigm of participatory action research within a PhD structure 
and it is important to record the ethics structures of the organisations involved. In every case study 
explored in this thesis, the ethics process either had inherent ‘grey areas’ or some study team 
members felt ethics processes were incompatible with the principles of the participatory action 


research paradigm. While there are difficulties in mapping this within the structure of a PhD, this 
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section attempts to report on these incompatibilities. This is an important part of the learning 
process and, thus, of the results. The implications described in this section are explored in more 


detail in the Discussion chapter. 


Codifying and co-labelling relevant power structures from the start of the research process ensures 
that the very concept of a PhD must also be scrutinised according to the values of those working in 
partnership. This is especially relevant when working with people at greater risk of exploitation. For 
example, Australian Indigenous communities who have traditionally been over-researched and 
under serviced,” people affected by rare diseases, and those populations of special interest to 
genomics research. In this doctoral research, these communities are represented in the Indigenous 
Precision Medicine case study, the ausEE case study and the Shared Ancestry case study, 


respectively. 


The study teams worked with each of the communities using the participatory action research 
paradigm. Under this paradigm, it was essential to map any ethical issues identified by community or 
study team members and to work in partnership to co-define the issues and co-create solutions to 
bridge any ‘disconnects”.4®34*4 This included addressing complex questions such as power and 
representation in communities, *®? which is particularly complex for genomics research, as findings 
might have implications beyond individuals. There are also tensions in balancing participants’ 
autonomy while ensuring that they are protected from risks they may not be aware of — such as 


their data being used for purposes for which they have not given consent.’ 


Shared Ancestry 


The STARDIT Alpha report for the Shared Ancestry case study was co-created by study team 
members and participants and provides important data.” It noted that “the co-design process 
took longer than expected owing to ethical ‘grey areas’, with no clear instruction on whether ethics 
approval was required to involve people in co-design”.* As a result, an after the initial ethics 
approval, subsequent feedback from the co-design process was integrated by applying to modify the 


ethics approval to take into account the preferences of potential participants. 


During the online discussions, participants identified ethics committees as having “failings”, and one 
participant asked “who will decide who will be on the ethics committee?” [P4-SA]. Another 
suggested that members of the sibling group could “form and seek out participants for the Ethics 
Committee” [P6-SA]. Another added “I am not sure of the ethics process, but it does seem a shame 


that more of us cannot participate” [P5-SA]. One participant stated “I trust the scientists and the 
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ethical committees” [P9-SA], while another noted that “the ethics of DNA research generally will 
continue to be of huge importance and will continue as a political issue, triggering new laws and 
regulations” [P7-SA]. This person raised a concern that “the law will not be able to keep up with the 
research — and we as members of the general public won't either” [P7-SA]. One participant noted 
that while people might be experts about a process or data, that does not make them “moral 
guardians” [P5-SA]. This person concluded that “no single body (and that includes the church and 


the government) has a right to dictate moral guidelines” [P5-SA]. 


During the Shared Ancestry project, it became apparent that some current ethics processes are 
incompatible with a participatory action research paradigm. The following is a short extract from the 


STARDIT report written after completion of the project: 


The entire study team agreed that limitations in the ethics process affected the extent of 
how the sibling group could be involved in the study. Internationally, confusion still 
surrounds what ethical approval is required before involving potential participants in co- 
designing research.*”° On the advice of the La Trobe University Human Research Ethics 
Committee, the study team did not approach potential participants about co-designing the 
study until after ethics approval had been granted, with feedback from participants being 
incorporated by a number of subsequent modifications to the original ethics application. As 
a result of the complex process of modifications, the timeline for feedback was shorter than 
the study team had anticipated, although the process did provide useful feedback. 
Ambiguous policies for the ethical involvement of people in co-designing research can 
hamper the degree of control potential participants have in research and further clarity from 


ethics committees will enhance power sharing at this crucial stage of research. 23” 


An unplanned change of Chief Investigator (Cl) and administrative processes related to the ethics 
process (which were outside of the control of the study team) led to the facilitator (MC) reporting 
that support from outside the study team was “non-existent” and was “wholly inadequate” for the 
participatory research process being used. 7?” For example, investigators were instructed by the 
outgoing Cl that they could not contact participants at all, including to explain that the study had 
been suspended. This removed participants from the ‘participatory’ loop and confirmed the La Trobe 
University Human Research Ethics Committees as a locus of power, which is incompatible with the 
principles of the participatory research paradigm in which everyone is involved at every level.‘ As 


all of the participants of the Shared Ancestry project are biological relatives of myself (JN) (including 
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my mother), as one of the investigators of the study team this created unique challenges, which 


were recorded in the reflexive research diary | kept during the research process. 
ausEE 


Potential participants were involved in co-designing the study and concluded that involving children 
affected by EGID would be too ethically complex. Consequently, only those over 18 years old were 
involved in co-design and participation. Like the Shared Ancestry project, the co-design process took 
longer than expected owing to ethical ‘grey areas’, with no clear advice from the ethics committee 
on whether ethics approval was required to involve people in co-design. As a result, after the initial 
ethics approval, subsequent feedback from the co-design process was integrated by applying to 
modify the ethics approval to take into account the preferences of potential participants. Again, the 
study team agreed that the limitations of the ethics process affected the extent to which potential 


participants group could be involved in study co-design. 
ASPREE 


In the ASPREE case study, participants’ preferences to have their data returned to them presented 
ethical difficulties for the proposed multi-generational research study (MGRS). In a response to an 
email survey, the lead investigator for ASPREE-XT (PL) stated that it was “fascinating” that all 

participants wanted “a self-managed future of health information”, which he described as “very at 


odds with the current system” and requiring “justification to the ethics committee”. 


In addition, some participants had concerns about themselves or their biological relatives (especially 
offspring) finding out information they “might not want to know” (P6-ASPREE), or having it imposed 
on them. This issue is common to almost all ongoing and proposed genomics research studies and 
should be urgently addressed by contemporary ethical frameworks, which favour individualist 


perspectives over collective ones.?”° 


Indigenous Precision Medicine project 


This initiative encountered difficulties in publishing a protocol about a planned co-design process, as 


ethics approval was required even before starting the co-design process. 


During the co-design stage, participants raised concerns that information from research data 
gathered for the project could be used for law enforcement purposes, and that researchers could 
not guarantee that it would not be. The research protocol responded by combining genomes to 
create a reference genome, rather than using a biobank model in which individuals’ data were 


recorded. 
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Sharing data and publishing 


There are related ethical issues with regards to sharing research data and publishing participatory 
action research in peer-reviewed journals. The United Nations‘ Report of the International Bioethics 
Committee on updating its reflection on the human genome and human rights’ explores scientific 
knowledge about the human genome. It characterises it as a “heritage of humanity”, states that it is 


“a common good”, and that “open access should be therefore guaranteed”.°° 


Data from genomics research require continuous evaluation of the scope of informed consent, 
including the balance between confidentiality, data protection, and equitable sharing of research 
results with all participants.” This point was echoed most articulately by ausEE case study 
participants. Four ausEE participants mentioned insurance as an area of concern that requires public 
involvement and scrutiny with regards to data sharing, in particular in relation to data being misused 
by for-profit companies. One participant stated that “progressive watering down of privacy 
protections” meant they were concerned their children would be “refused insurance because of a 


decision | have made” [P16-ausEE]. 


The United Nations report also calls on the international community, governments and researchers 
to address the very complex issue of the conflict between the right to access scientific knowledge 
and other relevant principles, including “patents for protection of intellectual property” .”™ This issue 
was also raised by a participant in the Shared Ancestry case study who, unprompted, stated that 
“Research is for humankind. Its benefits should be available to all. Information should be for the 
most part easily available. If it's publicly funded, it must be publicly available. Patents should be 


limited in time, and patenting new life forms is unethical” [P9-SA]. 


Emerging forms of ‘citizen’ or ‘participatory research’ mean that people are increasingly empowered 
to “collect their medical information and even to analyse it”.°° Some ausEE case study participants 
commented that people affected by diseases exist “across economic and political boundaries” and 
that access to medication is affected by “who owns, profits, controls, patents” [P3-ausEE]. These 
concerns were echoed in the United Nations report, which states that the trend for limiting access to 
research data “is fostered by the secrecy of the pharmaceutical companies and academic medicine 
about the results of their research, so that individuals with rare diseases are using the existing 
resources like the internet and Citizen Science to find out more about their conditions and diseases 


and share this knowledge”.°° 
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Access to information, including ‘borderless’ access, was raised as an issue that affects who can be 
involved in genomics research [P3-ausEE]. The participants from the ausEE case study also stated 
that paywalls for articles have direct negative impacts on patients, carers and parents who wish to 
make informed decisions and “judicious choices” about healthcare, or to get involved in research 
{P3-ausEE]. Participants from both the Shared Ancestry and ausEE case studies shared the view that 
if public money is paying for research, participants should have control of these data, and research 
about participants should not be published behind paywalls, which participants often cannot afford 


to access. 


The STARDIT system was developed in partnership with the WikiJournals, a Wikimedia Foundation 
project that provides free access to both public and peer-reviewed research. While STARDIT was 
used to report aspects of this doctoral research in the public domain, financial tensions exist as most 
open access publishers charge researchers fees to publish open access articles. For example, 
universities encourage PhD candidates to publish in journals as they are perceived to be prestigious 
or have a high ‘impact factor’. This is often motivated by universities’ desires to meet academic 
outcome measures, which have a strong connection to their business models, which in turn may 
reflect policies and performance measures of governments and other funders. In addition, within 
universities there are often not accompanying budgets to pay open access fees.** This situation 
makes it difficult to publish results from participatory action research in a format that participants 


themselves can freely access. 


Academic structures can thus create competing, conflicting or even perverse incentives to publish 
research behind paywalls in order to meet perceived prestige thresholds. While there is nothing 
inherently wrong with publishing in pay-walled journals because they are considered ‘prestigious’, in 
the context of the participatory action research paradigm, these arrangements raise the question of 
whether the priority is sharing knowledge openly, or sharing it in a way that is perceived as 


beneficial to the host or funding organisation. 


A further advantage of STARDIT relates to sharing of data about the ways in which participants were 
involved in initiatives. Creating STARDIT in partnership with WikiJournals and other organisations 
allows people to self-identify with certain ‘labels’ when describing how they were involved in 
research, and the preference mapping tool provides a way to map different stakeholders’ 


preferences for data sharing and publishing. 
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Chapter 10 — Discussion 


Introduction 


In this chapter, | will: 

e summarise the aims and main findings of the thesis (see Table 10.1) 

e summarise the outputs from this thesis 

e evaluate and appraise the research design, methods and limitations of this thesis 


e synthesise some findings from the results and discuss their relevance to communities of shared 


interest 
e summarise new knowledge generated by this thesis and dissemination of 
findings,demonstrating how some findings and outputs from this thesis are already being used 


e discuss the implications of the findings and new knowledge for genomics research and other 


related areas 


e summarise my recommendations based on these findings, including areas for future research. 
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Table 10.1: Summary of aims and findings 


Aims 
To understand when and how 
people have been involved in 


human genomics research to 


date. 


To apply a participatory action 
research (PAR) method to 
human genomic research using 
four case studies, in order to 
learn more about the 
practicalities of involving 


people in genomic research. 


To develop a standardised way 
of planning, reporting on and 
evaluating involvement 
(STARDIT) in order to improve 


future genomics research. 
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Activity 


Reviewed the literature to assess methods of public 
involvement that may be applicable for human 
genomics research (or those that are already being 


used). 


Applied a participatory action research paradigm to 
studies of four different groups of people associated 
with human genomics research (underway or in 
planning) and used participatory action research to co- 


produce STARDIT. 


Planned and conducted the involvement activities and, 
where appropriate, shared learning experiences, and 
described how involving people affected or informed 


future genomic research. 


Initiated and then led the facilitation of the process with 
multiple international stakeholders. Co-created a 
standardised data sharing system to compare the views 
and perspectives of different stakeholders about 


genomics research across the four case studies, 


Output 


Published scoping review, with 
20 citations (as of 1* November 


2021, as per Google Scholar). 


Three published case studies 
and a published research 
protocol for a genomics 


research project. 


(1) An Alpha version pre-print 
(cited 5 times as of November 
2021); (2) published a public 

consultation report about co- 


design of STARDIT Beta;7*? (3) 


Findings 


Improved understanding of contemporary 


involvement in genomic research. 


Lack of a standardised way of reporting involvement 
in human genomics and of transparently evaluating 
ways people are involved, has resulted in no 


evidence base to inform best-practice. 


Participants reported positive impacts from 
involvement in all four case studies, in addition to 


important learning relevant to each community. 


Stakeholders from all case studies reported changed 
views regarding involving people in research as a 
result of either participating or being a study team 
member. All changed views represented a move 
towards valuing the importance of involving people 


in genomics research. 


STARDIT provides a useful way to report data about 
genomics research and other initiatives, including 


data about how different stakeholders are involved. 


STARDIT is useful beyond genomics research. 
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including views about involvement, methods of 
involvement, and any impacts and outcomes of the 


involvement. 


published a working Beta 
version Standardised Data on 
Initiatives (STARDIT);? (4) and a 
final version submitted for 
review (under review as of 


November 2021).® 


Statement of main findings 


In this thesis, | have demonstrated ways of: 

1. mapping and reporting how current genomics research initiatives are involving people, 
including presenting this as a scoping review 

2. mapping different stakeholders’ preferences for involvement (researchers, participants and the 
wider public) 

3. involving potential participants in designing how they will be involved in the research process 
and reporting these data 

4. reporting how stakeholders have been involved in the research process using standardised, 
structured data which is both human and machine readable 

5. comparing standardised data about involving people in order to evaluate the processes and to 


help inform future decision making for those planning to involve people in research. 
Scoping review 


The scoping review improved understanding of contemporary involvement in genomic research. The 
review has been well read at 7929 views, and has been cited by 20 other peer-reviewed papers and 
five Wikipedia pages (as of 1* November 2021, as per Google Scholar). Learning from the scoping review 
and the methodological techniques developed has informed subsequent scoping of reviews in the 


field.?? 


The scoping review found that that “without a standardised framework to report and transparently 
evaluate ways people are involved, it will be difficult to create an evidence base to inform best 
practice”. This finding informed the initiation and subsequent co-creation of the ‘Standardised Data 


on Initiatives’ (STARDIT) framework. 
Standardised Data on Initiatives (STARDIT) 


Co-created in parallel with the case studies, STARDIT provided a consistent way to report the 
planning, methods and evaluation of the case studies. This work facilitated a cross-case analysis that 
provided standardised quantitative and qualitative data and allowed comparisons between the case 
studies to be made. The STARDIT system also facilitated the collection and comparison of impact 
data, including impacts from transformative learning, which would otherwise have been challenging 


to record, report and compare. 


Case studies 


Participants enjoyed the experience of being involved in the case studies and participating in the 
online discussions, telephone interviews and face-to-face events. The reported impacts and the fact 
that participants enjoyed taking part in the online discussions have added relevance in a post-COVID- 
19 world, in which use of online tools to involve people in accessible and evidence-informed ways 


will be a priority for genomics initiatives and other kinds of initiatives around the world. 


Participatory action research (PAR) proved to be a successful paradigm to guide the research 
processes. However, participatory action research methodology was sometimes incompatible, or 
practically difficult to achieve, within existing ethical frameworks. The implications of these results 


are discussed in detail in the section ‘Implications for ethics and participatory action research’. 


Positive impacts and important learning relevant to each community were reported from all case 
studies. Stakeholders from all the studies reported changed views about involving people in research 
as a result of either participating or being a study team member. Where quantitative data were 
gathered, all changed views represented a move towards valuing the importance of involving people 


in genomics research. 


Study team members also reported changed views as result of involving people. For example, some 
ASPREE study team members who reported being hesitant about involving people at the start of the 
process, reported changed views about the value of involving people as a result of being part of the 
process. While such ‘transformational learning’ is a valuable outcome, it is extremely difficult to 
capture and express in quantitative data. This demonstrates the importance of both learning by 


doing and using platforms such as STARDIT to capture that learning. 


The STARDIT platform provides a way to record data about impacts and outcomes both 
quantitatively and qualitatively. However, the skills and knowledge required to plan, do and report 
involvement will require further research into the learning and development needs of stakeholders 
for each project. Mapping the knowledge and skills of stakeholders, and any areas where these 
might need development, can be done by using STARDIT-PM as part of the cycle outlined in the 
STARDIT Beta version (Figure 3.5). This provides ways to embed learning and development needs 
into the design of a project, to report on needs assessment, and to evaluate the impact of any 
learning interventions. For example, application of STARDIT-PM might identify that certain 
communities are currently not included in any planned involvement activities. The mapping process 
might identify that study team members need to develop their communication and community 


outreach skills or knowledge of the community in order to involve a community more effectively. 


Any learning and development intervention to improve these skills and knowledge, any impacts or 
outcomes of it, could be reported using STARDIT. Being informed about impact data from previous 


projects may help people planning future research to make evidence-informed decisions. 
Outputs and dissemination 


The published outputs from this thesis have collectively received more than 43 citations, 


demonstrating the interest in the issues explored in this thesis. 
Review of Public Involvement in Global Genomics Research 


This review has been cited 20 times (as of 1* November 2021, Google Scholar),”” and the methods of 
categorising tasks, methods, stages of involvement have been adopted by other reviews. The data 
extraction methods and findings from this review also directly influenced the inception of 


Standardised Data on Initiatives (STARDIT). 
Standardised Data on Initiatives: Working Beta version 


The STARDIT platform has helped plan and evaluate different projects, showing impacts from the 
processes. STARDIT is already being tested by the Wikipedia-integrated open access peer reviewed 
WikiJournals, which has articles which are integrated into Wikidpedia.*® This include STARDIT being 
used by the Wiki Journal of Medicine to report data about an article on systematic reviews and 
scoping reviews. STARDIT has been used by other research initiatives beyond this PhD thesis, and 
has been recommended for reporting involvement in biobanks.* People from multiple organisations 
continue to be involved in the co-creation, including people from Cochrane, Johns Hopkins 
University, Health Research Authority UK, European Organisation for Rare Diseases, Australian 


Citizen Science Association, The Poche Centre for Indigenous Health, and multiple Universities. 
STARDIT adopted by Australian Genomics 


Organisations beyond the partner organisations from this doctoral research have also used the 
STARDIT Beta version. The organisation Australian Genomics has established a working group to 
establish ‘Involve Australia’, an organisation which aims to “produce a set of health public 
involvement guidelines and resources for genomic research, prioritising the public perspective” .7“° 
The project also plans to “conduct learning and development interventions to ensure genomic 
researchers learn how to effectively involve the public in their research”.””° | was invited to be a 
member of the working group, which voted in favour of using STARDIT and created a prospective 
report to describe their planned work. The report will be updated throughout the lifetime of the 


project, and represents the first example of STARDIT being used by a national genomics organisation 


to report planned involvement of the public and patients in their work. 


Work in Australian Government health technology assessment 


During my PhD candidature, | was appointed to the Medical Services Advisory Committee (MSAC) by 
the Australian Government Minister for Health.*®” | have given multiple presentations at five 
meetings since 2019, giving presentations about the public interest and patient perspective on 
medical technologies, including genetic testing. My own personal experience of bereavement when 
my partner died from sudden adult death syndrome in her early twenties (from what was likely an 
inherited cardiac rhythm disorders), informed my response, along with my professional experience 
from this doctoral research. The feedback | have provided has influenced Australian Government 
health technology decisions in relation to the use of genomic testing and related diagnostic tools. 
This included contributing to the public summary document for an item about ‘Genetic testing for 
diagnosis of inheritable cardiac rhythm disorders’.©? During meetings and in associated 
documentation, | made the point that as well as allowing better clinical diagnosis and access to 
potentially life-saving treatments, free genetic testing may also allow people who currently do not 
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have treatment options to ‘join communities of shared interest’°’, a concept informed by my own 


personal experiences, and my work on this doctoral research. 
Voluntary work for Cochrane 


| was invited to be part of both the Cochrane Advocacy Advisory Group, co-opted onto the Cochrane 
Consumer Network Executive and elected as a member of the Cochrane Council. In these roles, | 
have been able to inform Cochrane’s work in relation to both involving people in genomics research 
and reporting involvement in systematic reviews using STARDIT. In addition, | have been invited to 
co-lead work on developing a Cochrane Values Statement. The co-design process for this work is 


being reported by Cochrane using STARDIT. °38 
Voluntary work for the Wikimedia Foundation 


The Wikimedia Foundation hosts open-access peer reviewed journals, which are Wikipedia 
integrated. As well as being a member of WikiJournal of Science | was elected as the Strategy Liaison 
for the WikiJournals. Working in these two voluntary positions, | was able to work with the Editorial 
team to co-design STARDIT and integrate it into the WikiJournal system, using WikiData. As a result, 
STARDIT is already being tested by the WikiJournals. This includes STARDIT being used by the 


WikiJournal of Medicine to report data about an article on systematic reviews and scoping reviews.?® 
Presentations and contributions to public debate 


| have given a number of in person and online presentations, including webinars, PhD 
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presentations*®, and winning prizes in two Three Minute Thesis competitions. My entry for the 2018 


‘Visualise Your Thesis’ competitions was used as part of the learning resources for the two online 
discussion case studies and has received over 500 views. °% | have also been invited to give two 
presentations to the Board of the Poche Centre for Indigenous Health to present about both my PhD 
findings and about STARDIT, in order to inform future reporting of genomics research by the Poche 
Centre. | was also invited to give a presentation to Ludwig Boltzmann Gesellschaft (Austria) in 2020 


on participatory action research, informed by my PhD.*9?*92 


| have informed public debate, by participating in discussion on the UK’s Genetic Society’s podcast as 
a guest, discussing my PhD and STARDIT. The podcast was the most listened to episode of the 
Genetic Society’s podcast in 2020. In summary of my contribution on the podcast, a review from 
BioNews (part of the Progress Educational Trust) stated ”the public must be given more power, both 
in how this research is conducted and how this data is used”.*% It continued “it's time for us as the 
public to get involved. If you would like to expand your perspective on one of the most relevant 


scientific breakthroughs of the past century, this podcast is well worth a listen”. 


Table 10.2: Summary of presentations during doctoral research 


Title 


Invited by 


Given by 


Date 


Standardised Data on 


WikiCite 2020 Virtual conference 


Jack Nunn and 


27" October 


Initiatives (STARDIT)*4 Thomas Shafee | 2020 
(recording, 
resource and 
transcript) 
Standardised Data on Poche Centre for Indigenous Jack Nunn 18 
Initiatives (STARDIT)*>_| Health, 9th Annual Research November 
Showcase Program 2020 
Involving people in Poche Centre for Indigenous Jack Nunn 8t September 
DNA research Health, Research Advisory Board 2020 
Involving People In Ludwig Boltzmann Gesellschaft Jack Nunn 24th 
DNA Research (recording, September 
resource, 2020 
presentation) 
Genomics Research La Trobe University Jack Nunn 13" October 
and Involving People*®? 2020 
Involving everyone in Australian Citizen Science Jack Nunn 1* April 2021 
research: Creating the | Association 
evidence*”° 
Involving People in Rare Voices Australia Jack Nunn 5th August 
Rare Disease Research 2021 
Science World Evidence-based healthcare | Jack Nunn 20th October 
Communication in a day (recording) 2021 
Pandemic 
STARDIT and Industry Genomics Network Jack Nunn 25th October 
InGeNA Alliance 2021 
Ideas Forum - Stem Stem Cells Australia Jack Nunn 9th 
Cells Australia December 
2021 


| have also co-produced two podcasts in my capacity as Director of Science for All which discuss 


using STARDIT to report participatory action research, and how it integrates into WikiData. 


Synthesis of ideas 


Through synthesising knowledge from the literature review and learning from the participatory 
action research process used for the four case studies, this section will answer the following 


questions: 


1. What did we learn from involving people in genomics research, that we would not otherwise 


have known, and that could inform future genomics research? 


2. Is enough being done to involve different stakeholders in shaping genomics research? If not, 


what should be done? 
Ways of protecting people from exploitation in genomics research 


The findings from this thesis have particular relevance for people at greater risk of exploitation. In 
this thesis | describe working with three populations at greater risk of exploitation, to map their 
preferences about involvement and to involve them in research. Members of the first population 
share a common sperm donor. This large group has potential to provide new research insights into 
medical, genetic, sociological and psychological studies. As they are of inherently high interest to 
research, they are at greater risk of exploitation by researchers and those looking to profit from 


discoveries made about the population. 


During my candidature | discovered that | was part of this population, and initiated the co-design 
process described in the case study. After that was complete, this population of half siblings was 
approached by a research organisation (which will not be named) inviting them to participate in 
some genomics research. Participants were asked to either give consent to participate in one 
particular study or have their tissue and DNA stored “for future use in research projects that are an 
extension of this research project”. As the consent document put it, “Alternatively, we may use your 
sample for future research that is closely related to the original research project or as a control 


tissue sample”. 


This invitation came after some of the siblings had participated in the online discussions for this 
doctoral research project. In the subsequent STARDIT report, the Shared Ancestry Case study 
participants recorded that the online discussions had helped them make an informed decision about 
participation in the study. In some cases, participants decided not to share their DNA and tissue 
samples beyond the scope of the specific project they were invited to participate in. This is an 
important example of a significant real-world impact from a co-created learning intervention that 


facilitated and moderated discussion. Such interventions, and the processes they can enable, can 


help people at greater risk of exploitation explore their own preferences about research and make 


more informed decisions about participation. 


The second population | worked with and describe in this thesis comprised people affected by 
Eosinophilic Gastrointestinal Disorders (EGID), including Eosinophilic Oesophagitis (EoE), which is a 
rare disease.” These participants were concerned that research priorities were driven by a desire for 
a return on investment. Four people mentioned insurance as an area of concern requiring public 
involvement and scrutiny in regard to data sharing. As already stated, one participant commented 
that it is a “valid community fear” that research data will be used “for” insurance companies or other 
for-profit organisations [P3-ausEE]. Participants’ concerns about misuse of data by insurance 
companies, governments and industry indicate a power imbalance. They clearly demonstrate the 
importance of involving potential research participants in making decisions about how their data will 
be shared and who will make decisions about data access. Both the STARDIT-PM tool and STARDIT 
reports provide standardised, human and machine-readable data about people’s preferences, 
empower people to create a public record of such preferences, and hold researchers and other 


stakeholders to account should those preferences not be respected. 


The final population | worked with and report on as part of this thesis are potential participants of 
the Indigenous Precision Medicine initiative, which is ongoing. In common with Indigenous peoples 
worldwide, they can be considered at greater risk of exploitation than other populations.” The 
reasons for this are explored in detail in Chapter 7. The study team’s decision to adopt STARDIT as 
the reporting system for the protocol indicates that it is valued by researchers working on co- 
designed genomics research with Aboriginal peoples. By reporting the planned research, and how 
the project plans to involve people, the study team have created a public record to which people 
from the Aboriginal communities can refer and, if necessary, hold researchers and other people from 


dominant groupings or cultures to account.” 


Learning from the three completed case studies has implications for involving people in research 
around the world. This learning is particularly valuable in developing countries and countries where 
people may not be literate (or ‘health literate’) or able to give informed consent to participate in 
research. Creation of public records on how people have been involved in research, how data will be 
used, and who is involved in decision making may allow people at greater risk of exploitation to 
make more informed decisions about participation in research. Just as there are social determinants 
of health,” there are social determinants of involvement which can weaken or strengthen “people’s 


control over the factors that affect their health” 7% 


The scoping review of involvement in genomics research conducted for this doctoral research 
identified a number of government-led initiatives that have reported on recruiting research 
participants from populations at greater risk of exploitation.”” For example, initiatives in the USA and 
UK sought to improve recruitment to genomics research initiatives by involving participants who 
identified as members of specific ‘ethnic’ communities. Genomics England published the following 


advertisement:*?? 


Become a Genomic Patient Champion: In order to help tackle health inequalities in some 
communities, we are keen to hear from people who are involved in community-based work 
or activities. Could you draw upon your experience of participation to offer support and 
advice about the 100,000 Genomes Project within community groups such as Black and 


Minority Ethnic (BME)? 


This advertisement is similar to others used by government-led genomics research initiatives in UK 
and USA. Of concern, these often combine the concepts of ‘ethnicity’ and cultural communities. 
While involving people from diverse communities in recruitment can be effective at improving 
recruitment,’ leveraging existing partnerships or community engagement solely to increase 
participation raises concerns. Where people are only involved in recruiting others to research (rather 
than in other aspects such as governance) this might raise ethical concerns. Similarly, the questions 
of who defines ‘communities’ and whether people self-identity as members of them (such as the 
‘Black and Minority Ethnic’ ‘community’) have important implications, in particular for countries with 


a history of systemic racism, including Australia.^53454 


If lessons from the Belmont Report on ethical principles for research involving human subjects are to 
be heeded,** to ensure that people are not exploited, involving them in recruitment must be done 
alongside involving them in other aspects of the research. Involving people can help ensure power 
imbalances do not exist and that individuals’ and communities’ health priorities are met. Vigilance 
from the public, policy makers and researchers is needed to ensure that the concept of 
‘involvement’ is not reduced to a method of improving recruitment in populations at greater risk of 
exploitation, rather than of sharing power with them throughout the research process. Systems like 
STARDIT can empower more people to be involved in creating, sharing, analysing and using data to 
help inform decision making and, ultimately, to improve both their own health and healthcare 


systems. 


People’s decisions to participate in research (either publicly or privately funded) may be influenced 


by public domain data generated from STARDIT reports and similar platforms. Governments and 


health service providers who are paying third party providers to store and access data can now 
create public domain reports about how data will be shared, sold or analysed. By extension, this has 
important implications for people around the world who are paying for-profit companies to provide 
direct-to-consumer DNA testing. Often these services use a ‘tick-box’ consent process or the 
requirement for users to consent to sale of their data to third-party organisations are embedded in 
user agreements. Systems like STARDIT can empower anyone in the world to create transparent 
reports about how such data are being used or planned to be used. Using machine learning and 
other analysis tools, potential consumers of DNA testing from for-profit companies may be able to 
make informed decisions about which service they choose. These decisions could be based on 
publicly available data that show how each company will use DNA and other data, making it easy to 
compare the practices of competing services. The availability of such data may prompt companies to 
improve their consent processes so that people continue to share their data. The STARDIT and 
similar platforms may thus empower those who are currently disempowered by relatively 


unaccountable companies operating in international free markets. 


Strengths and limitations 


The aims of this thesis were: 


1. To understand when and how people have been involved in human genomics research to 
date 

2. To apply a participatory action research method to human genomic research, using four case 
studies, in order to learn more about the practicalities of involving people in genomic 
research 

3. To develop a standardised way of planning, reporting and evaluating involvement in order to 


improve future genomics research 


While all of the aims of this thesis have been achieved, the purpose of this section is to evaluate the 
strengths and limitations of different paradigms and methods used within the thesis to achieve 
these aims. As this thesis was guided by a number of paradigms and used a number of different 


methods, this section is divided into sub-sections which evaluate each paradigm and method. 
Strengths and limitations of paradigms used 


This doctoral research was guided by a number of paradigms, which in themselves guided the 


theoretical frameworks and methods. This section is a short evaluation of those paradigms. 
Evaluation of the rights-based paradigm 


The application of the rights-based paradigm was helpful during this thesis. The detailed 
documentation provided by the United Nations, World Health Organisations and other organisations 
provides a clearly articulated framework on which to both justify and build a participatory action 
research process. By relying on internationally ratified frameworks, it releases the individual 
researcher from the burden of both providing and justifying their own values, and asking others to 
agree. Concepts such as universal human rights are so universally accepted, that they are thus a 


helpful starting point to frame all other discussions. 


While the tension between ‘consumer’ rights, ‘individual human rights’ and concepts of ‘collective 
solidarity’ were naturally impossible to solve with doctoral research, the methods described here 
provide a foundation to provide exact and multi-lingual descriptions of actions, conflicts of interest 
and labels to which people might self-identify. STARDIT can be used to report these kinds of data in 


relation to the participatory action research process. This data, when combined with data about 


impacts and outcomes, can have an empowering effect and help promote individual, consumer and 


collective rights in the context of genomics research and beyond. 


When working with Indigenous peoples in countries such as Australia where there is both a history 
and continued evidence of systemic racism and structural violence towards Aboriginal peoples*°?***, 
international paradigms such as the UN ‘Declaration on the Rights of Indigenous Peoples’ provide 
helpful guides when co-designing a participatory action research process, and a helpful reference 
when both Federal and State level legislation, policy and funding is lacking.2 The use of the STARDIT 
system to also publicly state how the participatory action research process would work, including 
shared control and Aboriginal people involved in data access decisions, means there is a public 
record which any research participants can use to hold researchers to account, should the protocol 
waiver from the original co-defined principles. STARDIT reports may provide a useful way of building 
trust between researchers and genomics research participants at risk of exploitation (such as 
Indigenous peoples), by providing a clear record of how research has been designed, and with who. 
When people are asked to provide informed consent to participate, such information can provide 


clear and unambiguous information about every aspect of such a study, and how it aligns with both 


FAIR” and CARE?” data sharing principles. 


Applying the open access paradigm meant that all of the papers in this thesis were shared using 
open access publishing. This was reported in detail in the section ‘Sharing data and publishing’ of the 
Results chapter. It is important to note there can be difficulty in achieving this when University 
budget policies (including that of La Trobe University) state that PhD budgets cannot be used for 
open access fees. This can mean that PhD researchers are forced to either publish behind paywalls, 
or seek alternative means of funding, which is not always possible. Publishing the peer-reviewed 
articles from this thesis in open access journals relied on me volunteering as a member of the 
Editorial board for a journal and funding from another University, in order to access free publishing. 
The debate about open access publishing and who should pay for it is wider than the scope of this 
thesis*?®>?1, but has serious implications for participatory action research led by communities with 
limited financial resources to pay open access feed. The Wiki Journals offer free peer-review and 
publishing of articles, and such volunteer-led models, backed by large charities to provide IT 


infrastructure, might be one such viable model. 


While the rights-based paradigm is a helpful starting point, it is worth noting that such frameworks 
are still limited by language, and are often attempts to express concepts, values and moral stances 
which are not easily expressible in language, or where important meanings can be lost when 


translated. While ‘real-world data’ about applications of such rights frameworks in healthcare and 
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beyond can help articulate concepts°”, working to co-define such concepts across multiple human 


languages remains an ongoing challenge in this paradigm. 
Evaluation of the participatory action research paradigm 


This thesis was guided by the paradigm of participatory action research where possible. While it was 
not always pragmatic and was ethically complex to apply the paradigm in the context of a PhD, the 


participatory paradigm guided every aspect of the thesis and is evaluated in this section. 


Informed by the participatory action research paradigm, we created a private online space for the 
facilitators from both case studies which used online discussion, in order to provide a place for self- 
critical reflection, share experiences, and offer support to one another. Feedback from facilitators 
from both case studies was that this online space was extremely helpful and provided both practical 
and emotional support. For examples, technical issues with using the online platform could be 
quickly solved with group discussion. Similarly, issues with participant engagement could be 
discussed and compared across the two studies, providing facilitators with reassurance that a 
relatively slow reply rate was occurring in the other online discussion too, and that it wasn’t their 


own facilitation that was causing this. 


While the co-design process worked well, it was inherently difficult working within academic 
structures and attempting to do true participatory research . For example, the ethics processes 
within universities are not designed for participatory action research and constrained the process by 
requiring the design to be submitted before people were involved, which resulted in needing to do 
formal modifications multiple times for each different study, in order to incorporate feedback from 
the different stages of the participatory action research process. Allowing sufficient time for co- 
design was also challenging and limited the extent to which participants could be involved in some 
tasks, such as co-authoring papers. Planning in more time for this process would have improved the 


duration given to stakeholders to give feedback as part of the participatory action research process. 


Two of the case studies were originally part of the same ethics application, but on the advice of 
University staff, they were divided into two different studies, with the data collection, questions, and 
discussion structure remaining almost identical in order to allow a cross-case analysis. Designing 
case studies in parallel meant that in order to have some consistency (for purpose of comparison) 


some aspects were inflexible regarding co-design. 


The participatory action research process for STARDIT worked very well. Facilitating this process 


through the legal structure of the charity Science for All (auspiced by the Royal Society of Victoria) 


gave more methodological freedom to the participatory action research than would have been 
possible within the structure of a university. For example, organising face to face events was done 
according to the organisational policy, and covered by public liability insurance of the Royal Society 
of Victoria. However, while the participatory action research process used to develop STARDIT was 
enormously helpful’, the scope of STARDIT grew beyond just genomics research. While it remains 
useful and relevant for planning, reporting and evaluating involvement in genomics research (as 
demonstrated by the uptake by Australian Genomics)”, it can now also be used across multiple 


other disciplines, increasing the likelihood of any future uptake. 


Involving people affected by research in how it is planned and conducted is best practice. 
Accordingly, from the outset of my doctoral research | began to establish a formal advisory group to 
guide the overall doctoral research (in addition to my supervisory team, a formal academic team 
which guides doctoral research on behalf of the University) in order to incorporate the perspectives 
of people directly affected by genomic research, such as people with diseases caused by genomic 
variation. However, this endeavour was actively blocked by some members of the supervisory team, 
and thus I could not proceed with this way of working. While it is impossible to estimate how this 
decision may have affected the overall doctoral research at the review stages, | was able to conduct 
more informal (and unpaid) involvement of people affected by genomic research, including 
conversations with people affected by rare diseases which influenced the design of elements of the 
doctoral research (see the section ‘Unselected case studies’ for more information). The subsequent 
use of the participatory action research paradigm to guide each individual case study also ensured 
that all research with the various communities from the four case studies actively involved people 
directly affected by genomics research (or who could be affected), including people affected by 


variations of known significance and communities of people with shared ancestry. 
Strengths and limitations of methods used - summary 


Using more than one theoretical approach is preferable in health research as no one single 
methodological framework will provide all the data and insights required in order to answer the 
complex questions proposed by this thesis.“ It is noted that each chapter contains an analysis of 
the strengths and limitations of the methods within. Accordingly, Table 10.3 summarises an 


evaluation of the multiple methods used in this doctoral research. 


This section relating to STARDIT in the table below is an evaluation how effective STARDIT was for 
enabling consistent data reporting and a cross-case analysis. It is not an evaluation of STARDIT itself. 
A discussion about the strengths and limitations of STARDIT can be found in the discussion section of 


the peer-reviewed section ‘Standardised Data on Initiatives - STARDIT: Beta Version.’® 


A more detailed evaluation can be found in the appendix, in the section ‘Strengths and limitations of 


methods used — detailed analysis’. 


Table 10.3: Summary of strengths and limitations of methods 


systematic reviews: 
How are the public 
involved in health 
research and what are 


the impacts? 


complete a meta-analysis of quantitative or 
qualitative data about involvement in genomics 


research. 


2: Informed decision that a systematic scoping review 
was the most appropriate method to search for 


relevant data. 


3: Findings suggested that methods of involving 
people guided by the paradigm of participatory action 


research were most likely to have impacts. 


Method Section Strengths Limitations and future research 
evaluated 
Reviews Narrative review of 1: Demonstrated there is not currently enough data to | 1: No systematic reviews specific to genomics research 


exist. 


2: Number of systematic reviews in this area was 


limited at the time of the review. 


Public Involvement in 
Global Genomics 
Research: A Scoping 


Review 


1: Database provided by the Global Alliance for 
Genomics and Health (GA4GH) provided a useful 
‘snapshot’ of current international genomics research 


projects. 


1: Authors were aware of initiatives not represented in 


database. 


2: Updated database after the completion of the review 
added 125 new initiatives”, creating an impetus for an 


updated review. 


Method 


evaluated 


Section 


Strengths 


Limitations and future research 


2: A decision on whether an initiative was reporting 
involvement required individual judgement, which 


was checked by multiple authors. 


3: ‘Involvement indicators’ were developed an 
integrated into STARDIT so methods were more 


repeatable. 


4: A subsequent review exploring reporting the impact 
of involvement in biobanks cited this review, used a 
similar method, and recommended the use of 
STARDIT for reporting the impact of any 


involvement.?2 


3: The review extracted data at the level projects 
articulated in the database and did not analyse the 
structure of organisations which sat above multiple 


projects. 


4: The search was related to the wider issue that 
concepts such as ‘involvement’ cannot always be 


expressed fully in linguistic constructs. 


5: It was not always possible to determine whether 
involvement was ‘consequential’, so an assumption was 
made that all methods reported resulted were 


‘consequential’. 


Guidance for planning, 
reporting and 
evaluating initiatives: A 
multidisciplinary 


scoping review 


1: This review explored guidance for planning, 
reporting and evaluating initiatives beyond the scope 
of genomics research in order to find relevant models 


and frameworks.*%2 


2: This review showed the current variation in 
guidance on planning, reporting and evaluating 


initiatives in order to inform both future systematic 


1: This review was carried out by a small team, so the 
capacity of the team limited the search design to 
certain methods which were pragmatic, rather than 
systematic. For example, we did not search the citations 
of results from the database search owing to the limited 


capacity of the team. 


Method Section Strengths Limitations and future research 
evaluated 
reviews and proposed standardised ways of reporting 
data on initiatives. 
Online 1: While the research projects described in the case 1: Some participants logged-in multiple times and read 
discussions studies in this doctoral research were planned and comments but did not comment. While the follow-up 


completed before the COVID-19 pandemic, the 
methods of involving people online described in this 
thesis now have an unexpected relevance to many 
disciplines, as research projects around the world seek 
to involve people online in novel ways, and evaluate 


such methods in a standard way. 


2: The ASPREE study showed that people preferred 
not to use social media companies to get involved in 
research, and this method aligned with the data 
storage expectations of participants (safe, secure and 


not owned by third party company). 


3: The online text-based discussion method itself 
provided a flexible way for people in different time 


zones to get involved and interact. 


survey attempted to capture the views of participants, 
not everyone responded, so it is hard to understand the 


behaviour patterns of those who did not comment. 


2: One study team member noted the difficulty in 
achieving ‘the balance of being prescriptive’ (for 
consistency) and giving freedom to facilitators to 


initiate discussions and follow emergent themes. 


3: Two participants from both online discussion case 
studies reported finding the platform ‘complicated’ and 


problematic [P5]. 


Method 


evaluated 


Section 


Strengths 


Limitations and future research 


4: The facilitators reported that the support they 
received improved their ability to facilitate 
discussions, as did the shared online discussion space 
for the facilitators from both the Shared Ancestry 


study and the ausEE study. 


5: Owing to how data was reported and shared 
(including using STARDIT), such learnings from these 
online discussions can be used to inform others who 


are planning similar methods. °° 


Case study 
and cross- 


case analysis 


1: The case studies selected for this doctoral research 
represent four unique and real-world communities of 
people, where genomics research affects their lives 


directly. 


2: By ensuring that communities of people affected 
were involved in shaping this research, it has helped 
ensure learning is anchored in reality, rather than 
theoretical models of involvement in genomics 


research. 


1: The sample sizes for the case studies in this thesis 
were variable, as was the percentage of people 


recruited from the known populations. 


2: While datasets are still too small to draw any 
Statistically significant conclusions, the mixed-methods 
approach meant that the interpretative analysis was 
able to combine themes and provide useful data, 


including comparing differences and generalisations. 


Method 


evaluated 


Section 


Strengths 


Limitations and future research 


3: The case study methods used in this thesis, guided 
by the paradigms described, allowed the collection 
and analysis of both qualitative and quantitative data 
from multiple sources, which provided a richer 


dataset. 


4: The collection of data from study team members 
and participants provided a more holistic perspective, 
and meant data and impacts were collected that 
would not have been if data was just collected from 


participants. 


5: Using STARDIT to combine data demonstrated a 
way to overcome this variation in both sample size 
and percentage of people recruited in the case 


studies, by combining standardised data. 


6: While the case studies were variable in size and 
each had both ‘representative features’ and ‘deviant 
features’ (see the section ‘Case selection’), the most 


significant learning for others planning genomics 


3: The selection of case studies was influenced by 
pragmatic considerations. It is important to note that 
the unselected case studies also occupied a ‘work-load’ 
grey area, with conversations with potential partner 
organisations identifying areas for support in the 


organisations | was approaching 
Future research 


Future combinations of data could articulate more data 
to help understand any causal relationships for what 
‘widened’ people’s preferences for who should be 
involved. For example, data about whether it was being 
involved in the process, or a specific learning and 
development resource which changed people’s 


perspective. 


Future applications of data collection using STARDIT 
could provide this level of articulation, which was not 
possible within the scope of this doctoral research. Such 


data could aid machine learning to help establish 


Method 


evaluated 


Section 


Strengths 


Limitations and future research 


research was likely to be from the ‘generalisable’ 


features of the case studies. 


7: The cross-case analysis provided a successful way to 
apply the post-positivist paradigm and explore 
common themes across the case studies, and combine 
the quantitative data to allow cross-case quantitative 


analysis. 


evidence informed ways of involving people with the 


most impact. 


Standardised 


data reporting 


Standardised Data on 
Initiatives - STARDIT: 


Beta Version 


1: The preference mapping tool (STARDIT-PM) allowed 
consistent mapping of different stakeholder 


preferences across all case studies where it was used. 


2: Applying a quantitative analysis to the identical 
questions asked at the start and the end of the online 
discussion case studies allowed an investigation of 
baseline preferences about involvement in genomics 
research and preferences after the online discussions. 
This enabled an analysis which showed how people’s 


preferences changed, including showing a ‘widening’ 


1: The co-creation process for STARDIT occurred during 
this doctoral research, with Alpha versions initially used 
to report. While Alpha reports were converted to Beta 

reports, Alpha reports were only published as PDFs, not 


structured data, requiring manual data extraction. 


Method 


evaluated 


Section 


Strengths 


Limitations and future research 


towards people preferring more kinds of stakeholders 


involved in genomic research. 


3: Data was published online in the public domain ina 
consistent way, allowing it to be combined in the 
cross-case analysis, and open to future datasets which 


might use the same question structure. 


4: STARDIT allowed consistent data on case studies to 
be reported (including self-reported outcomes), 
facilitating any future analysis and allowing future 
statistical analysis to begin to draw any correlation 
between certain learning interventions and learning 


outcomes, which may suggest causality. 


Outcomes 
and impact 


assessment 


1: While time for longer term impact assessment is 
not possible within time limits of a PhD, it was still 
possible to measure outcomes and impacts 
immediately after the participatory action research 


process. 


1: A follow-up survey was conducted with the study 
team for the ASPREE study, but not the participants, 
owing to ethical restrictions. As a result, there may be 


multiple unreported impacts. 


Method 


evaluated 


Section 


Strengths 


Limitations and future research 


2: Some impacts were able to be measured over two 
years after the online discussions had finished, as 
participants from the Shared Ancestry study were 
involved in co-creating the Alpha version STARDIT 
report in October 2020, and able to edit the Beta 


version from August 2021 onwards. 


2: Preference and follow-up data was also not collected 


for the ASPREE study for the same reasons. 


New knowledge 


This section contains a short summary of the new knowledge generated from this thesis, which is 


explored in more detail in each respective chapter. 


The scoping review showed that there was inconsistent reporting of involvement in genomics 
research. This included the finding that there is no standardised way to report on how people with 
certain lived experiences (for example, mental health phenotypes) have been involved in genomic 


research. 


The scoping review and case studies also revealed the importance of involving multiple stakeholders, 
especially study team members and staff working with people affected by diseases as part of health 
services and research (not just the ‘public’ or patients). The perspectives of people with a 
professional relationship to those affected by diseases can provide valuable insights for research 
design. This is especially the case when potential research participants are affected by neuro- 
degenerative diseases; are too young; or unable for other reasons to be involved in co-design 


processes. 


The case studies and the subsequent cross-case analysis showed that involving people in the 
participatory action research process has multiple impacts, including a ‘widening’ of people’s 
preferences about who should be involved in research to include more people: potential research 
participants, members of communities of shared interest defined by genomic variations, health 


professionals and research study team members. 


The STARDIT system was informed by both the scoping review and the participatory action research 
methods used in the case studies. It has been demonstrated as a proven way to report on the 
preferences of all stakeholders and on planned or completed participatory methods. By providing 
both a standardised way to report on involvement, and a way to report how people with certain 
lived experiences of phenotypes have been involved in an initiative, STARDIT addresses gaps 
identified in the scoping review. Such data, combined with data on methods of involving people, on 
modes of communication, and on tasks people were involved in, can be combined with impact data 
to help those planning involvement in research make evidence-informed decisions about research 


design. 


Implications and recommendations 


Learning from this doctoral research thesis has implications within and beyond human genomic 
research, with applications in both wider genomics research, participatory research methods and 
citizen science across all disciplines. This section discusses the implications of this doctoral research 
on genomics research and beyond, and embeds the associated recommendations. As this thesis 
concerns human genomic research, the recommendations concern only the multiple domains of 


human genomics research (including research translation and health technology assessment). 


The recommended use of systems such as STARDIT for reporting in these domains are numbered as 


follows: 


1. report how people with lived experiences have been involved in annotating phenotype data 
2. report how people have been involved in designing, managing and evaluating genomic research 
3. report genomics research with populations at greater risk of exploitation 


4. report health technology assessment processes around the world, including evidence 


assessment processes and how different stakeholders have been involved. 


It is further recommended that STARDIT is used to: 


5. transparently plan and report international debate on youshéng and eugenics. 
Implications for future human genomics health research 


The unique features of genomics require new evidence-informed methods for informing 
involvement and power sharing in research. Tools such as “CTRL” ”* offer genomic research 
participants more control over how they participate in genomics research. CTRL is an “internet- 
based platforms to create a communication interface to support ongoing participant-led 


management of their involvement in research studies”.”4 


Systems like STARDIT can be used to report how different stakeholders will be or have been involved 
in genomics research. Combining platforms like CTRL with reporting systems such as STARDIT will 
allow research participants, potential research participants, research funders and the wider public to 
appraise how their data will be used. Systems such as STARDIT can therefore be used by individuals 
wishing to make informed decisions about participating in genomic research or sharing genomic 


data, including participating in biobanks. As it provides a system to report genomics research and 


involvement in a standardised way, STARDIT reports can help people make informed decisions about 
participation in research, for example, by allowing potential participants to view data about who is 

involved in data access decisions and any financial ownership of data. Such standardised information 
about genomics research can also be used by research funders or potential research funders in order 


to make decisions which align with any organisational values or policies. 


STARDIT data can also be used to help assess the methods of genomic analysis, including who was 
involved in articulating how people with lived experiences have been involved in annotating 


phenotype data. 


Recommendation 1: Report how people with lived experiences have been 


involved in annotating phenotype data in a standard way 


The 2019 scoping review highlighted the importance of being able to show who was involved in 
labelling phenotypes for genomic variations where a subjective lived experience is important.’ For 
example, subjective experiences of depression or dementia can only be articulated by the person 
experiencing them. The better our understanding of such subjective lived experiences, the more 


useful such data are when combined with genomic data. 


In 2018, in the ‘The Power Threat Meaning Framework’, the British Psychological Society stated that 
“we need to take meaning, narrative and subjective experience seriously” in relation to defining, 
diagnosing and treating mental health.” The WHO estimates that 264 million people suffer from 
depression, with 800,000 people annually committing suicide. °°° Suicide is the second leading cause 
of death in 15 to 29-year-olds.°” Similarly conditions such as dementia are recognised by the WHO 
as global research priorities.°°° Accordingly, there is a new imperative to combine data about the 
subjective lived experiences of people with more objective measures such as genomic analysis, 


medical records and data from standardised assessments (such as those used in the ASPREE study). 


The methods used in the case studies for this thesis show how participatory action research 
processes informed by post-positivist and constructivist theoretical frameworks can integrate 
subjective perspectives into research. Systems such as STARDIT can be used to report who will be or 
who was involved in such data articulation, and allows people to self-identify with particular 
groupings or categories. Compared with relying solely on a diagnosis by a healthcare professional or 
on objective measures, this provides more power in the process of constructing data to describe 
lived experiences.**? Involving people in the co-creation and structuring of such data will enable 


better quality data on phenotypes to be produced. 


The findings of this doctoral research support other studies which suggest the need for more 
opportunities to actively involve people affected by genomic variations in the process of articulating 
phenotypes.” Involving people in reclassifying clinically ascertained variants of uncertain 
significance (VUS) can “impact risk assessment, medical management, and psychological outcomes 
for patients and their families.”°°’ The STARDIT system uses structured data (including standardised 
taxonomies and ontologies) to facilitate the articulation of common human experiences in multiple 
languages. It thus enables more people around the world to be involved in annotating phenotype 
data. In addition, STARDIT can work across cultures and human languages. This means that concepts 
such as ‘depression’ can be described in different cultural contexts and different languages, where 


both factors can significantly affect people’s descriptions of these concepts.” 


Using systems such as STARDIT to report on how people with specific genomic variations self- 
identify with standardised categorisations (such as a person with dementia or depression) can help 
ensure there is more machine-readable data about genomic research. In turn, this will help inform 


future reviews and appraisal of data quality in this field. 


Recommendation 2: Report how people have been involved in designing, 


managing and evaluating research in a standard way 


More research is required to appraise methods of co-designing and co-managing biobanks and other 
genomic research.”’ In coming decades, methods for managing biobanks will need to be constantly 
reviewed and appraised by all stakeholders to ensure cost effectiveness and accountability. The 
STARDIT system has been recommended as a way of reporting involvement in biobanks, and such 
transparent reporting should be urgently explored by all current and proposed biobanks and 


genomic research.?”34 


Recommendation 3: Report research with populations at greater risk of 


exploitation in a standard way 


As described in the section ‘Ways of protecting people from exploitation in genomics research , 
STARDIT can be used to plan, report and evaluate genomics research in order to ensure that 
research is workable, culturally safe, acceptable and effective. STARDIT should also be used to report 
on how people from populations at greater risk of exploitation have and will be involved in research, 
allowing more people from such populations to be involved in the process, including reporting and 


evaluating how people were involved. 


Implications for health technology assessment 


What is health technology assessment? 


Health technology can broadly be defined as the pharmaceuticals, devices, procedures and 
organisational systems used in health care. Health Technology Assessment (HTA) is a process used 
by governments and health insurers to assess which ‘health technology’ to pay for in certain 
populations and circumstances. The process usually considers the medical, organisational, economic 
and societal consequences of implementing health technologies or interventions within the health 
system.°°? Assessment processes and the quality of assessments vary by country, however, all 
require decision making based on the available evidence. As physical resources are always finite, 
governments worldwide use HTA as part of a ‘rationing’ system — that is, to decide on and justify 


allocation of resources to health technologies. 


In the context of HTA, the definition of ‘evidence’ is elusive as assessment of any health technology 
requires an individual subjective judgement based on the available data. Health Technology 
Assessment can therefore be understood as the product of systematic observation or experiment.°”? 
In order to provide practical and useful knowledge, it often relies on data collected and analysed in 
accordance with a pre-established protocol.°” As part of HTA processes, it must be agreed what 
kinds of data and evidence will be integrated, how they will be critically appraised, and if and how 
different kinds of data will be ‘weighted’ in the analysis. It is important to note that while HTA 
processes frequently uses the word ‘evidence’, the word ‘wisdom’ is often lacking.*2° As summarised 
by Menon, health technology policy making “cannot be reduced to a technical exercise, free from 


values or ethical judgement” .°1? 
Problems with health technology assessment and genomics research 


A number of difficulties inherent in the HTA process have specific relevance for the translation of 
genomics research into public health genomics and other health interventions. As previously 
discussed in this thesis, it is difficult to collect and integrate data about subjective lived experience 
(including data on patient reported experience and outcome measures). This means that evidence is 
often drawn from more objective data sources, such as data on deaths. Recognition that people with 
variations of known or unknown significance need to be involved in the HTA process is essential for 
ensuring that people with lived experience of variations are involved as a distinct stakeholder group. 
When drafting a public involvement strategy for the UK Health Research Authority in 2013, | 
successfully argued that “those with known genetic dispositions” should be categorised distinctly 


from the general public and patients.*8*°” 


There can also be inherent inequity in the application of health technologies to genomics research. 
In small groupings of people affected by rarer variations, lack of statistical ‘power’ means that ‘gold 
standard’ evidence will never be available for HTA decision-making processes.*>?° Notwithstanding, 
some authors have claimed that most published research findings are false and “flexibility in designs, 
definitions, outcomes and analytical modes” increases the chance that findings are false.°*? In 
Australia, nearly 50 per cent of deaths from cancer are from rare cancers, yet a US review of 
oncological drugs in 2015 suggested that current pricing models reflect “what the market will 
bear”.°”°4 Ensuring equity in the translation of genomics research into health technologies to treat 


‘non-profitable’ rare diseases is a significant challenge.*?° 


Financial or other interests can also increase the likelihood that research findings are false. For 
example, a 2017 Cochrane systematic review found that “sponsorship of drug and device studies by 
the manufacturing company leads to more favourable efficacy results and conclusions than 
sponsorship by other sources” .^”? In addition, ‘asset exchanges’ of patient groups between 
pharmaceutical companies may also occur when lobbying during the HTA process is led by patient 
groups but funded (often indirectly and not always transparently) by pharmaceutical companies. 
This creates potential for the most profitable diseases to be the ones approved and funded as a 
result of HTA processes.*?° Such inherent bias in the research done, the results published, and 
financial support for lobbying (which includes financial support to give feedback from the patient 
perspective during complex HTA processes) affects the quality of the evidence available to those 
making HTA decisions. In the scoping review for this thesis, the novel finding of a method of 
articulating variation in the perspectives of people affected by rare diseases provides a foundation 
for developing what may become an increasingly important model for involving these populations in 


every stage of genomic research, including HTA.2” 


Population screening of people’s DNA using knowledge from genomics research has already been 
modelled as cost effective in Australia. ° However, there is also an inequity issue when translating 
genomics research for population screening for Indigenous peoples. For example, in Australia, 
Aboriginal peoples are under-represented in current genomics research, so the analysis of genomic 


data may not be as useful for these populations.° 
DIY-Bio and health technology 


As access to medicine (including genomic medicine) is not universal or affordable, the Do-it-yourself 
Biology (DIY-Bio) movement needs to be considered when looking at the future of HTA processes.* 
As access to technology to modify human genomes or apply genomic medicine grows, the United 


Nations is investigating governance challenges.°*® As for any innovation, there are risks and 


opportunities. Inclusion of the DIY-Bio movement into public health genomics has potential to 
provide access to genomic medicine for millions, as both the discovery process and creation of tools 
for genomic medicine are shared as part of this open-source movement. Health technology 
assessment processes will need to carefully consider questions of quality control and licensed “good 
manufacturing practice” (GMP)>2° to ensure they remains inclusive and do not only include those 


who can traditionally afford it, such as large pharmaceutical companies. 


The risk of counterfeit, ‘off-licence’ or poor-quality genomic medicine being used by people who 
cannot afford access to better regulated interventions is very real. Systems like STARDIT can be used 
to report on the entire process (including manufacturing), any licenses, any competing interests, and 
other important data. This can help ensure that HTA processes include genomic medicine and 
related technology, regardless of whether it is a small community-controlled enterprise or an 


international company.*22 


Recommendation 4: Use STARDIT to report health technology assessment 
processes around the world, including evidence assessment processes and 


how different stakeholders have been involved 


Health Technology Assessment processes around the world need to make sure that people are 
involved using evidence-informed methods that require standardised reporting of data about how 
people were involved and any impacts. Additionally, data beyond simple health outcomes data 
(death date) and economic evaluations are required to integrate subjective lived experiences more 
effectively. Improving data about quality of life and other important outcome measures identified by 
stakeholders is important. Standardised reporting of data about financial or other interests is also 
needed alongside other research data — this includes interests such as being personally affected by a 
disease (as articulated by the ausEE case study participants) and any financial support that patient 


organisations are receiving. 


The STARDIT system allows sharing of standardised data about research methods, analysis, 
outcomes and multiple stakeholders’ interests and allows both humans and machines to appraise 
the research studies using such meta-data. The STARDIT system also allows reporting of data about 
participants involved in research or in the HTA process, including any genomic variations or diseases 
with an underlying genomic cause. Using STARDIT to plan, report on and evaluate HTA processes 


would improve evidence-informed decision making worldwide. 


Implications for public discussion debate on the concepts of 
youshéng (eugenics) 


Defining youshéng and eugenics 


The One Health model of public health proposed by veterinarians raises no objections to humans 
selecting for perceived favourable characteristics in other animals based on DNA analysis. However, 
when applied to humans most agree that such principles are ethically complex. The comparison of 
selective breeding for non-human animals and humans is not made lightly. The ‘First International 
Eugenics Congress’ took place in London in 1912 and was organised by the British Eugenics 
Education Society.” It was attended by Sir Winston Churchill and presided over by Major Leonard 
Darwin (son of Charles Darwin). In his opening remarks, Darwin said that the principles of better 
breeding are known by farmers, and it would require “courage” to apply such principles to 
humans.°”° He said “might not we hope that the twentieth century will in like manner be known in 


the future as the century when the eugenic ideal is accepted as part of the creed of civilisation” .°7° 


In Anglophone countries, the word 'eugenics' comes from the Greek word ‘Eu’ for ‘good’ and applies 
to concepts of perceived genetic quality, reliant entirely on subjective judgement. Because of its 
association with racist applications, the word in English is highly loaded with negative associations. 


Academics divide the concept of ‘eugenics’ into two categories:°7? ‘ 


negative eugenics’ and ‘positive 
eugenics’ (which it must be noted are not value judgements, but descriptions of the mechanism of 
‘away from’ or ‘towards’ certain perceived genetic qualities). In summary, positive eugenics 
describes a method of trying to encourage reproduction between people with perceived positive 
phenotypes. Negative eugenics describes trying to prevent reproduction between people with 


perceived negative phenotypes. It is important to note that both positive and negative eugenics can 


be forced on people, or entered into by choice (‘opt-in’). 


In Mandarin Chinese, the word for the concept of eugenics is ‘youshéng xué’ (hereafter shortened to 
youshéng). In Mandarin Chinese, the word ‘lacks cultural baggage’, compared with the English word 
‘eugenics’, which is most often associated with forced eugenics.” Recognising the associations of 
the word ‘eugenics’ in English, the term ‘youshéng’ is used in this discussion. The Mandarin Chinese 
term has positive connotation related to ideas about giving birth to children of ‘better quality’ or 
having a ‘healthy birth’ .*?25231 For example, smoking during pregnancy would not be considered 
youshéng.°~* Some contemporary Chinese ethicists have controversially argued that, as members of 
society, individuals “have a duty to provide society with healthy and normal children” .°23(") 


However, as the Chinese Government has ratified the United Nations Convention on the Rights of 


Persons with Disabilities, it is obligated to reform laws that conflict with this convention.°24°)) 
Accordingly, clinics licensed to do pre-implantation genetic diagnosis are only permitted to do so to 
avoid ‘serious disease’ or as an infertility treatment. Selection for other traits (including gender) is 


not permitted.°2? 


Examples of forced eugenics 


An example of negative eugenics occurred in the USA in the 1920s when comparisons were drawn 
between the social imperative of vaccination and compulsory sterilisation.°7°"°29?) This application of 
negative eugenics by the ‘state’ purported to protect its ‘interests’ by sterilisation of those 
considered to be “afflicted with an hereditary form of insanity or imbecility”. °° This was challenged 
in the 1927 United States Supreme Court case ‘Buck v. Bell’, and provides an important perspective 
from which to view the origins of removal of autonomy in public health genomics. As recently as 
1993, the Japanese Government forcibly sterilised citizens for similar reasons, with the ‘Eugenics 
Protection Law’ only repealed in 1996.°’ In the United States, California only banned coerced 


sterilizations of female prisoners in prisons in 2014 (distinct from forced birth control).°78 


Examples of positive eugenics were promoted by organisations such as the British Eugenics Society, 
with Mary Barton and Bertold Wiesner (biological father of the participants of the Shared Ancestry 
case study in Chapter 7) among many other researchers publicly supporting the concept in peer- 
reviewed literature and stating in a 1945 British Medical Journal article that they only took donors 


from “intelligent stock” 4% 


One of the offspring of Wiesner, a participant in the Shared Ancestry study, shared examples of 
negative eugenics enabled by large technology companies such as IBM, which provided the Nazi 
regime with technology to carry out negative eugenics policies.” This person was concerned that 
this will continue to happen into the future. The Chinese Government's alleged collection of blood 
samples from ‘ethnic’ Uighurs (or Uyghurs) — and use of their DNA to map human faces in order to 
sort them into ‘ethnic’ categories — was initially facilitated by the technology company ‘Thermo 
Fisher Scientific’.°7° This alarming use of forced negative eugenics in genomics research has been 


described by the Canadian and USA governments as ‘genocide’.°7° 


Examples of contemporary ‘opt-in’ positive eugenics 


Prenatal chromosomal diagnostic testing has been available in some countries since the 1950s, °°° 


and health services around the world now routinely offer parents the chance to test foetuses for 
conditions such as Down syndrome. Testing methods have become increasingly sophisticated and 
less invasive (some now only requiring a small blood sample from the pregnant woman), and 


countries such as Australia are exploring routinely covering the costs of non-invasive pre-natal 


screening (NIPS) for Down syndrome, Edward syndrome, Patau syndrome and Turner syndrome.>*? 


In parallel with NIPS, people undergoing in-vitro fertilisation (IVF) can increasingly access pre- 
implantation genetic diagnosis. Such diagnosis can help prospective parents make informed choices 
about which embryo they will implant, and these decisions are often based on genomic variations of 
known significance. However, what is classed as ‘significant’ is a fundamentally subjective choice, 
connected with both concepts of youshéng and eugenics. Who decides who may make such 


decisions, and how, is central to the discussion of this thesis. 


In 2015 the United Nations International Bioethics Committee stated that: 


.. germline genetic interventions were the subject of science-fiction novels and scientific 
theoretical debate, but considered non-executable. That has changed ...This new reality calls 
upon experts, governments and all citizens to consider all the possible consequences on 


human rights and fundamental freedoms as well as on the future of humanity itself. °° 


At the start of this doctoral research in 2017, editing of the human genome was still an academic 
discussion. The international “scandal” surrounding the alleged first editing of the human genome in 
China in 2018 is a clear indication that the subject of positive eugenics and gene editing is no longer 
theoretical.°?? While parents may ‘opt-in’ to use such technologies, people who have had their 
genomes edited cannot ‘opt-in’, so it can be considered forced positive eugenics. The subject needs 
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urgent public debate, °°* and the UN has hosted recent public discussions to consider the “impact on 


the values and cultures of each society” of such technologies.”? 


Rapid advances in genomics analysis and falling costs mean that more people will soon have access 
to DNA analysis. This may allow them to make informed decisions about parenthood, including 
providing prospective parents and pregnant woman with more data when deciding whether to try 
for, or terminate, a pregnancy. Many people will seek guidance on these complex decisions from 
health professionals and genetic counsellors (with associated ethical oversight) or community 
leaders. Decision-making processes in countries such as Australia that are considering covering the 
costs of NIPS need to be transparent, including how decisions are made about which variations to 
test for, who made these decisions and how, and how it is decided who makes these decisions. For 
example, the views of communities with shared values (for example, communities defined by shared 
religion or faith or health economists) are not always compatible with the UN’s human rights 


paradigm. 


It is also important to note that while a country or culture might affirm individual choice in the 
matter of terminating a pregnancy, and the choice may appear to be available, in practice choice 
may be undermined by the social, political or clinical environment.*2>""22”) The ‘social model of 
disability’ argues that people are not disabled by impairments but by disabling barriers in 
societies.” For example, the choice of terminating a child with Down syndrome may be influenced 
by a perceived lack of support from society (including financial and practical support), rather than by 


eugenics, per se. 


For example, people with specific variations in the gene GJB2 often experience what is medically 
described as severe-profound hearing loss. However, from the perspective of some people affected 
by the gene, their deafness is a way of life, not a medical problem, and society disables them by not 
adapting to their needs.*?°'?) As the genetic causes of deafness are found worldwide, there is a 
global ‘deaf culture’, which has a strong sense of identity connected to living with hearing loss.°35(?>) 
Using the ‘right’ language to describe genomic variations, their known effects, and any responses or 
interventions to them requires in-depth awareness of both people’s personal experiences and of any 
sense of community that emerges when people share specific variations that affect their 


experience.??° 


Diverse perspectives are also evident in relation to having children with variations which may cause 
deafness. In order to make informed decisions, some people may wish to know more about 
inherited variations which may lead to deafness, and may seek information regarding carrier status, 
pre-implantation screening (IVF) or prenatal genetic testing. This information is important as 
according to a 2002 survey of over 600 people with hearing loss, some people stated they may 
choose to terminate a foetus that does not have specific variations that may lead to deafness, while 


others may prefer to terminate a foetus that does have such a variation.??*°” 


While seeking partners based on their DNA variations is an emerging area, a number of dating 
websites already claim to use DNA to find a ‘match’. It is not inconceivable that in the near future, a 
service may arise which allows people to share DNA variations they consider of significance in order 
to find partners they consider to be more compatible. For example, people who know they have a 
recessive variation known to contribute to a condition may seek partners who know they do or do 
not have such a variation. While this is currently a speculative service, questions of ethical oversight 


and subjective judgement would need careful debate. 


Recommendation 5: Transparently plan and report international debate on 
yoOushéng and eugenics 


While the history and future of youshéng and eugenics and genomics is beyond the scope of this 
thesis, | recommend that countries around the world urgently need to debate concepts such as 
youshéng in order to co-create ethical boundaries. Further, Anglophone countries need to start by 


considering how concepts of eugenics can be discussed. 


Systems like STARDIT can be used to map and report on the preferences and values of multiple 
stakeholders using standardised data about how any screening or testing decisions are made. Such 
reporting will allow transparent scrutiny of these processes (and of the values of those designing 
such processes). This has potential to create the enabling conditions for more people to be involved 


and to help improve decision-making processes. 


Similarly, initiatives to involve the public and other stakeholders in shaping policy, law, educational 
interventions and ethical oversight of the use of genomic technology can be transparently reported 
using STARDIT, allowing them to be evaluated. STARDIT could be used to map different stakeholders, 
their different preferences for involvement and their values. This data can be used to inform the co- 
design of any planned involvement activities such as: public debate; citizens’ juries and forms of 


public consultation. 


The looming shadow of future genomics research being used to reinforce racism and assist with 
genocide is haunting. Urgent public education, debate and international consensus must be 
established on these issues and, in particular, must ensure that people at greater risk of exploitation 


are involved. 
Implications for future genomics research beyond humans 


As humanity’s understanding of genomics (and other ‘-omics’) increases, the interaction between 
the environment, environmental systems, and other lifeforms (including the gut biome, flora and 
fauna) is being increasingly explored. This is summarised in the introduction to the 2008 ‘One 


Health’ report by the American Veterinary Medical Association, which stated: 


“The convergence of people, animals, and our environment has created a new dynamic in 


which the health of each group is inextricably interconnected.” °° 


A notable example of this convergence is the connection between the COVID-19 virus, human-to- 


animal transmission, and the social and environmental factors that created the enabling conditions 


for transmission. The importance of interdisciplinary research to both understand and solve such 


challenges is recognised by the World Health Organization.°?° 


Another example of this interconnection in the context of public health genomics was articulated 


clearly in a 2019 report by Genome British Columbia (Canada). It stated that: 


“our understanding of biological systems at a molecular level is transforming how society 
approaches solutions to complex challenges. These applications include health, forestry, 


fisheries, aquaculture, agrifood, energy, mining and the environment” .?”° 


A short video produced by Genome British Columbia explaining the concept of this interrelation 
was shared with case study participants in the online discussions as one of the learning 
resources.” The video was shortlisted by the study team from a ‘landscape analysis’ of education 


resources conducted by the Australian Genomics Health Alliance.*°° 


As humanity is part of Earth’s biological systems, there is a clear imperative for human genomics 
health research (including public health genomics) to be located within this wider context. The 
STARDIT system offers the only data sharing platform on which interdisciplinary research can be 
shared in a standardised way, across multiple human languages. The need for such a system of data 
sharing was reinforced at multiple stages of the STARDIT co-design process, with experts from public 
health, Johns Hopkins University, Cochrane, genomics research, environmental research and citizen 
science all involved and publicly supporting STARDIT. Its importance has been further highlighted by 
the COVID-19 pandemic. 


Implications for all research 


Those creating and delivering research, and those affected by it, may sometimes be the same 
people. Methods such as ‘citizen science’ and ‘participatory action research’ are blurring the lines 
between the concepts of ‘researcher’, ‘public’, ‘patient’ and ‘citizen’.2”°°226 Multiple stakeholder 
involvement (including the public; patients; communities defined by a shared interest; consumers of 
health services or medical products; payers; industry, and policy makers) is integral to ensuring that 
all perspectives are valued. The findings of the ASPREE case study supported this by demonstrating 
that learning involving research staff — not just senior researchers — was an important source of 


valuable perspectives.°® 


Some participants from the Shared Ancestry and ausEE case studies highlighted their preferences for 


online discussions, as they were more flexible and accessible and, thus, more inclusive. This has 


important implications for those planning involvement in future research, in particular, in the 


context of the COVID-19 pandemic, and the wider global adoption of working online. 


The STARDIT system provides a way to report data about research paradigms such as participatory 
action research processes and, regardless of the discipline, enables comparison of methods for 
involving people. The adoption of STARIT by both the Indigenous Precision Medicine project? and the 
Australian Genomics ‘Involving Australia’ project working group for reporting the initiatives, 


including planned stakeholder involvement.” 


Indigenous peoples around the world connect health and environment as linked and inter-reliant 
concepts.*4?>* Beyond the discipline of public health genomics, environmental research projects 
using DNA have also used STARDIT to report the initiative, including participatory action research 
processes. For example, a community-led environmental DNA research project used STARDIT to 
report the co-design and co-management process, and reported impacts.” Data about these 
initiatives indicated that using STARDIT allows their methods and impacts to be reported across 
disciplines, regardless of whether it is ‘health’ or ‘environmental’ research — helping bridge this 
conceptual gap in reporting initiatives, in a way that Indigenous peoples (and many non-Indigenous 


peoples) have done for millennia. 
Implications for ethics and participatory action research 


The findings of a 2018 scoping review about ethical challenges in community-based participatory 
research were supported by the findings of this doctoral research.*® In particular, questions of ‘who 
is an insider or an outsider’ and who decides this are central to the participatory action research 
process. 143145 This was particularly important in the case study undertaken for this thesis involving 
communities of shared interest defined by shared ancestry. A quotation from the Shared Ancestry 
study summed up this issue succinctly, with one participant asking, “who will decide who will be on 
the ethics committee?” [P4-SA]. The implications are similar for any genomics research which aims 
to recruit people from communities defined by ancestry, such as Indigenous peoples. Similarly 
questions such ‘who is an insider?’, ‘who decides this?’ and ‘what is the process for deciding who 
makes ethical decisions on behalf of others?’ are all questions which this thesis has demonstrated 


workable ways of answering, with a repeatable method. 


For example, publication of the protocol describing the intended use of co-design to co-create a 
genomics research protocol for working with Aboriginal Australians took longer than predicted 
owing to complex ethical review processes. Although some limited guidelines are now emerging?” 


from the UK’s National Health Service, internationally there is still confusion surrounding what 


ethical approval is required before involving potential participants as ‘specialist advisors’ in co- 
designing research.” The Indigenous Precision Medicine project embedded STARDIT reporting in 
the co-design process, including transparent reporting on how ethical decisions (including data 


access decisions) are made.? 


The thesis describes power imbalances in ethics processes which continue to affect genomics 
research around the world and impact negatively on the implementation of successful participatory 
action research processes.” While answering these questions will be complex and require ongoing 
consideration, it is clear from the case studies conducted for this doctoral research that the current 
situation is no longer sustainable. Arguably, current ethics processes lack accountability in some 


respects and are thus unethical. 


To ensure that power is shared effectively at all stages, these problems must be co-defined and 
solutions to them co-created and co-evaluated. The STARDIT-PM tool provides a way to report the 
preferences of all stakeholders regarding ethical oversight, and STARDIT allows these data to be 
reported in a standardised way. Potential research participants are able to add data to such reports, 
thus removing the power of researchers, who traditionally report research in peer-reviewed 


publishing models. 


While traditional ethics processes have been set up with the best of intentions, new methods are 
emerging which give people direct control over their own data in a very real way. An example is the 
CTRL model created by researchers working with the Australian Genomics Health Alliance.” This 


model allows people: 


to keep personal and contact details up to date; make consent choices (including indicate 
preferences for return of results and future research use of biological samples, genomic and 
health data); follow their progress through the study; complete surveys, contact the 


researchers and access study news and information.” 


Such tools offer an insight into future best practice. Among these, STARDIT offers a way to both 


report on the process and evaluate any impacts. 


Similarly, in the area of free market ‘consumer’ genomic data, where data are commodified for sale, 
blockchain technologies are giving people practical ways to be custodians and managers of their own 
data. Initiatives such as ‘Genecoin’ and ‘Encrypgen’ allow individuals to have control over their 


genomic data and gain financial rewards if they grant access to their data for research analysis.°*°°*” 


In contrast, people sharing genomic data with direct-to-consumer companies such as ‘Ancestry DNA’ 
and ‘23andMe’ simply ‘give away’ their data as an implicit trade-off for access to services.”®8 In 
addition, such companies often provide limited options regarding research participation, with some 


making data sharing with third-parties mandatory in order to use the service.*?°8 


To ensure initiatives have ethical oversight and protect people from exploitation, new research 
models are needed. Ensuring that all stakeholders have an opportunity to influence how initiatives 
are developed and managed (including data access decisions) will require the input of multiple 
experts and the implementation of complex co-design processes. STARDIT offers a way to plan, 
report and evaluate research processes in a standardised way. Such reporting is the first step in 
sharing power in genomics research in evidence-informed ways. This process will require continual 
evolution, including machine learning, to keep pace with the ethical challenges presented by 


genomics research. 
Implications for genomics research policy 


Organisations such as the Global Alliance for Genomics and Health which sets international 
standards for genomics research should urgently support standardised reporting of involvement in 
genomics research. Data from standardised reporting will support evidence-informed policy in this 
area in which public acceptability is central to ensuring the continued expansion of genomics into 
public health policy. Learning from the scoping review and best-practice examples — such as the UK 
Biobank and Genomics England — indicate the importance of ensuring long-term funding for such 
involvement. This includes funding to support stakeholder (including participant) involvement in 
tasks such as data oversight.”” Ongoing reporting and evaluation of such funding will allow policy 
makers to make evidence-informed decisions about the kinds of involvement methods, modes and 


tasks that are most cost-effective for payers, including tax payers and industry. 
Implications for policy relating to all research 


The processes described in this thesis have demonstrated a consistent way of reporting data about 
preferences for involvement, as well as standardised ways of co-designing and reporting on 
involvement in research. This has allowed comparison of preferences about involvement and a way 
of reporting and comparing the impacts of involvement. Policy and research funding criteria must 
begin to require standardised reporting of involvement so that initiatives can be evaluated in order 
to inform future research. The STARDIT system provides a way for initiatives to share data about 
involvement in research and to report impacts. This will invite scrutiny, promote transparent 


oversight of research and support evidence-informed policy. 


Chapter 11 — Conclusions 


In this thesis, | have demonstrated that there is a global imperative to improve involvement in 
genomics research, but there remains a lack of data to support evidence-informed policy in this area. 
| have led the design of several research projects with communities of people affected by genomic 
research and conducted case studies that explore and evaluate various methodologies for involving 


people in genomics research. 


Learning from these case studies, and the cross-case analysis, can be applied in different settings 
and with different population groups. These include populations at greater risk of exploitation, 
people affected by rare diseases, people who are of high-interest to researchers (such as large 
populations of donor-conceived siblings) and Indigenous sub-populations around the world. To 
facilitate involvement and data collection, | have also co-created methods of using a real-world 


online discussion platform and used it to involve people in every stage of the research cycle. 


My research has emphasised the importance of genomics research working globally across 
populations and human languages. Findings from the scoping review have shown that to date 
reporting of methods, including how stakeholders were involved in genomics research, has been 


both under-reported and of variable quality. 


In response, | have conceived and led an international co-creation process for a standardised 
reporting system (STARDIT) in partnership with people from multiple organisations including 
Cochrane, the Wikimedia Foundation and the Poche Centre for Indigenous Health and Johns Hopkins 
University. The STARDIT system provides a way to share data about involvement in genomics 
research and a system for reporting data about genomics research. It enables those planning 
genomics research to make evidence-informed decisions based on any data reported. In addition, 
STARDIT can be used to report on how people affected by diseases such as depression or dementia 
have been involved in research. It allows sharing of subjective lived experience of conditions and 


diseases which can improve the articulation of phenotype data. 


The STARDIT system can be used across human languages and cultures to co-create structured, 
machine-readable data which uses standardised taxonomy and ontology. The framework can also be 
used across other disciplines. This opportunity arises from the interdisciplinary nature of genomics 
research and the learning made possible by sharing information among disciplines using 
participatory action research processes and citizen science. As a multi-disciplinary system, STARDIT 


has relevance to describing initiatives in the context of global responses to urgent challenges facing 


humanity such as the COVID-19 pandemic and rapid climate change. The usability and relevance of 
STARDIT has been demonstrated by the use of the Beta version reporting tool by the Australian 
Genomics working group ‘Involve Australia’ in 2021, and by Cochrane for reporting the co-creation 


process of their ‘Values Statement.” 


In this doctoral research, | have used the participatory action research paradigm to guide the 
research process — including to co-define problems, co-create solutions and co-evaluate them — and 
to demonstrate practical ways of involving people in every stage of genomics research. Some 
fundamental ethical questions about power sharing have been uncovered during this research 
process, including the difficulties of achieving a true participatory model in an academic context 
where university ethics committees hold power in ways that can be incompatible with the paradigm 


of participatory action research. 


Recognising the limits of an academic research setting for the participatory action research method, 
| established and run a charity (Science for All) and have volunteered with organisations (including 
the Wikimedia Foundation and Cochrane) and worked for the Australian Department of Health in 
health technology assessment. This gave me the opportunity to learn and apply alternative methods 
of co-design and research management and to evaluate different approaches to participatory action 
research in research. Focusing on involving people in genomics research and standardising data 
sharing allowed me to combine learning from these projects with others around the world. This 
included learning about organisational governance, health technology assessment, collective 
decision making, power sharing, data sharing and co-evaluation processes. Working within these 
organisations provided the necessary foundations for learning about and applying the participatory 


action research paradigm, which is not always possible within academia. 


The most difficult work lies ahead, that is, translating the learning from this doctoral research into 
practice and continuing to ensure that data sharing about involvement in genomics research is 
standardised. Once achieved, humanity can begin to develop evidence-informed ways of sharing 


power in research that can be applied and evaluated by anyone, anywhere. 


Afterword 


“| realized that the difference that | saw between things was the same thing as their unity, 

because differences (borders, lines, surfaces, boundaries) don't really divide things from 

each other at all, they join them together, because all boundaries are held in common”? 

Alan Watts 

It has been an enormous privilege to complete this doctoral research. Along the way I’ve discovered 
new family from DNA tests, had the meaning of what ‘family’ means challenged, and involved them 
in research. I’ve spoken about my experience of losing my girlfriend to Sudden Adult Death 
Syndrome (SADS) to the Australian Department of Health in order to influence decisions on 
providing free genetic testing. I’ve worked with Aboriginal people in remote communities. I’ve 
learned that people from these communities have a deep understanding of the genetic causes of 
SADS. Working with them, we co-designed genomics research and explored what it means to have 
‘shared ancestry’ and ‘shared culture’. I’ve worked as part of a world-leading team, designing a 
multi-generational study, and I’ve worked with people affected by rare diseases and heard their 
concerns about data sharing. Every experience has taught me something, and I’ve valued each one. 


As | complete this doctoral research, it has reminded me why | have this passion for involving people 
in genomics research, which is effectively involving people in describing themselves. Because, of 
course, essentially we are all of these subjective lived experiences when it comes to mental health or 
any other part of our experience of being alive. Only we can describe this ‘phenotype’. 


| think my upbringing in western Europe and those traditions has made me come from an 
individualistic way of thinking about the human experience itself, which is often defined by the 
concept of the ‘self’ and the ‘Il’. However, this ‘I’ is challenged by genomics. Just as with issues like air 
pollution, extinction, or eco-system collapses — genomics research unites all of us because it affects 
all of us. Although we might think of genomics and DNA as essentially ‘us’ or ‘me’, it is of course 
shared with all life on Earth. The only way to make sense of it is to put it all in together and analyse 
it. This is where the importance in involving people in analysing this data is central. The potential for 
improving lives and ecosystems that genomics has is profound, but so also are the potential harms. 


Without involvement and power sharing, we get reinforcements of the social constructs of the 
dominant cultures, we get racist Al algorithms, we get linguistic divisions of populations based 
essentially on skin tone and a bit of geography. We get analysis that doesn’t tell the whole picture of 
genomics, it doesn’t include lived-experience, because often, it doesn’t ask to see it. 


Part of my PhD included online discussions with my half-aunts and uncles, some of whom identified 
as culturally Jewish and had Jewish ancestry. In a free and open discussion many of them mentioned 
events of the Second World War and the murder of certain minorities based on their perceived 
ancestry. They showed a genuine concern that genomics research is already being used to do this, 
and will continue to be used for this again into the future. 


We know from human history that the only way to protect us from this, is recognising that we’re 
actually protecting ‘us’ from ‘us’, there is no ‘this’. That's really what science education is. One of 
the original authors of what became the Universal Declaration of Human Rights was H.G Wells, who 
wrote “human history becomes more and more a race between education and catastrophe”. °° We 
need to learn how to save us from ourselves. We need to start to redefine ‘ourselves’ as meaning 
everyone and everything. While still in the early stages, | sincerely hope that STARDIT can be used to 
help everyone in the world share trusted knowledge, and provide the evidence we need on the most 
effective ways of saving us from ourselves, helping share the benefits of human knowledge with all 
life on earth. 


Jack Nunn, 16" March 2021, Melbourne 
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Appendices 
Methods 


Reviews 


How are the public involved in health research and what are the impacts? A 
narrative review 


Abstract 

Aim 

We wanted to understand how the public are involved shaping health research and summarize the 
established methods of involving the public that might be applicable to genomics research. By 


looking at any impacts that this involvement might have had, we wanted to identify possible 
methods and approaches that may inform our future plans for involvement and impact assessment. 


Method 


A systematic search of systematic reviews relating to public involvement in research was conducted 
using boolean operators in CINAL, Medline and Google Scholar. Data extracted from the reviews 
included what kind of involvement was taking place (the type), how it was done (the method), which 
stage of the research cycle it occurred and any impact on the research that might have occurred. 


Results 


This systematic narrative review summarises five systematic reviews identified in this area. 
Involvement was reported at each stage of research with mostly positive impacts reported at each 
stage. Involvement in data collection and analysis were the most reported stages of involvement. 
Agenda setting was the most frequently reported stage of the research cycle for involvement. Most 
of the methods of involvement described specified a method involving people in a group structure, 
rather than individually. A total of 27 different methods of involvement were extracted, with 
community based participatory research having the most impacts. Other than generic positive 
impacts, the most impacts reported were improving the relevance of the research, improving 
dissemination and improving data collection. The stages of research with the most reported impacts 
were ‘agenda setting’ and ‘data collection’, with dissemination the next highest. 


Conclusion 


Involving people in health research generally has advantageous outcomes for research and health 
service planning. Involving people as early as possible is considered best-practice. There are a 
number of high impact actions which can be taken which do not necessarily require considerable 
budgets or time. Face to face involvement appears to have an increased cost in relation to impact, 
compared to other forms of involvement, including involving people using online methods. Involving 
people in planning how the public will be involved is crucial. While there are limitations in the data 
available, it is clear that involving the people at any stage of research has value, with impact at all 
stages being possible if people are involved early in the research cycle. More work is needed to 
better document the impacts of involvement in research. 


Summary of narrative review 
Objective 


To summarise methods of public involvement in health research and any impacts these methods 
may have had, informing their applicability to genomics research. 


Background 


The is a growing body of literature about the methods of how to involve the public in health 
research, with a number of systematic reviews which summarise the methods and or the impacts of 
involvement in research. However, many reviews conclude there is limited data about involvement 
or low evidence of impact and are sometimes unable to draw strong conclusions. 

This narrative review attempts to summarise systematic reviews in this area to identify established 
methods for involving people and their and assessable impacts, to inform future plans for applying 
involvement and impact assessment strategies to genomics. 


Methods 

Search method 

A systematic search of systematic reviews relating to public involvement in research was conducted. 
Search terms 


A systematic search of systematic reviews relating to public involvement in research was conducted. 
The boolean operators for the searches were as follows: 


1. (Community+OR+Public+OR+Consumer*+OR+Patient*+OR+Carer*+OR+Volunteer*+OR+Adv 
ocate+OR+(Civil+society)+OR+user*+OR+Group*+OR+Citizen*+OR+Lay+OR+Population*+OR 
+Proband+OR+Residents+OR+Stakeholder*+OR+Carrier*+OR+client*+OR+survivor*+OR+par 
ticipant*) 


+AND+ 


2. (involv*+OR+engag*+OR+participat*+OR+consult*+OR+collab*+OR+contrib*+OR+(advisory 
+group)+OR+co-creat*+OR+co-produc*+OR+democra*) 


+AND+ 


3. research 


+AND+ 


4. (scien*+OR+biomedic*+OR+health+OR+clinical+OR+trial*) 


The search on CINAL returned 3,669 results. An identical search was carried out using MEDLINE and 
Google Scholar. 


Exclusion criteria 


Reviews which examined specific areas of research or specific populations were excluded. Similarly, 
reviews which excluded countries were excluded. 


Data extraction 


Data extracted from the reviews included what kind of involvement was taking place (the method), 
which stage of the research cycle it occurred and any impact on the research that might have 
occurred. 

Where possible, data was extracted in order to preserve any causal relationships. For example, if a 
review stated that ‘method X was used at stage 1, with the impact being Y’, this was extracted as 
one line of data. In some cases, there were no causal relationships to extract, so most analysis relies 
on the ratio that certain methods and impacts were reported. Text was initially copied verbatim, and 
was then coded into categories, using existing frameworks where possible. Impacts on the people 
involved and the researchers were also extracted but have not been included in this review. 
Similarly, data on barriers and facilitators of involvement has been collected but not included in this 
analysis. 


Results 


In total, 5 systematic reviews matched the search criteria. These are summarised in Table 1. 


Table 1: Included Systematic reviews 


Title Date | DOI 


Patient and service user engagement in research: a systematic 2013 | https://doi.org/10.1111/hex.12090 
review and synthesized framework 


policy and research, clinical practice guidelines and patient CD004563.pub28 
information material. 


Methods of consumer involvement in developing healthcare 2006 | https://doi.org/10.1002/14651858. 


Patient engagement in research: a systematic review 2014 | https://doi.org/10.1186/1472- 
6963-14-89 

A Systematic Review of the Impact of Patient and Public 2014 | https://doi.org/10.1007/s40271- 

Involvement on Service Users, Researchers and Communities 014-0065-5 

Mapping the impact of patient and public involvement on 2012 | https://doi.org/10.1111/j.1369- 

health and social care research: a systematic review 7625.2012.00795.x 


A number of systematic reviews had limited data regarding actual methods of involvement and 
reported limited evidence of impact. Where impact of involvement was reported it was almost 
exclusively positive. 


Variation in language 


Variation in language makes meaningful comparisons difficult?*?. 


Language to describe people involved in research 


In total over 46 different words and phrases were identified to describe people involved. The top 
four most common were variations on the word ‘user’ (including ‘patients and service users’), with 
‘community’ the 5* most used. 


Types of involvement (what) 


Involvement was reported at each stage of research with positive impacts were reported at each 
stage. Involvement in data collection and analysis were the most reported stages of involvement. 
While most methods of involvement did not explicitly state if it was group or individual, of those that 
did, more than twice as many specified that the method involved people in a group structure (31), 
rather than individuals (15). 


Tasks of involvement 


Influencing the design of the research was the most frequently reported task or ‘role’ (12), with 
interviewer (5), participant recruitment (5) and providing researchers with a perspective on the 
relevance of the research jointly second in occurrence. 


Stages of research 


In order to describe the stages of research, this review has adapted an existing framework. The 
most helpful framework was found in a review which describes patient and service user engagement 
(PSUE) at several research stages, within three larger phases: preparatory, execution and 
translational t. 


e Phase l: Preparatory Phase 
o Stage 1: Agenda Setting 
o Stage 2: Funding 
e Phase II: Execution phase 
© Stage 1: Study Design and Procedures 
o Stage 2: Recruitment and participation 
o Stage 3: Data Collection 
o Stage 4: Data Analysis 
e Phase III: Translational phase 
o Stage 1: Dissemination 
o Stage 2: Implementation 
o Stage 3: Evaluation 


Number of examples of involvement at each stage 


Using the framework above, the data was categorised to show the number of examples of 
involvement at each stage. This data is summarised in the Table 2 below and in Figure 1. 


Table 2: Involvement examples reported at different stages 


Figure 1: Number of examples at stage 


A 


wy 


m Preparatory Phase: Agenda Setting 


m Preparatory Phase: Funding 

m Execution phase: Study Design and Procedures 
m Execution phase: Recruitment and participation 
m Execution phase: Data Collection 

m Execution phase: Data Analysis 

m Translational phase Dissemination 

m Translational phase Implementation 


m Translational phase Evaluation 


Stages of involvement 


The stages of research mentioned in association with involvement most frequently were the 
‘Preparatory Phase: Agenda Setting’ (42), ‘Execution phase: Data Collection’ (36), ‘Translational 
phase Dissemination’ (24) and ‘Execution phase: Study Design and Procedures’ (21). 

Agenda setting and data collection were the stages with the most impacts (13), with dissemination 
the next highest (10). 


Methods of involvement (how) 


A total of 27 different methods of involvement were extracted, although it is likely that it does not 
reflect the diversity of methods used in all the studies included the systematic reviews as reporting 
of the methods of involvement is often imprecise or absent. For example, only 34 out of 385 lines of 
extracted data explicitly mentioned a method. Of all the methods, community based participatory 
research had the most impacts (2), with the method being reported as being used at all stages of the 
research. 


Impacts of involvement on the research 


Other than generic positive impacts, the most impacts reported were improving the relevance of the 
research, improving dissemination and improving data collection. It is unclear if more impacts were 
reported from dissemination and data collection because that is where people were most involved, 
or if that is because these kinds of outcomes measures are easier to measure. 


Methods and impacts — causal relationships 


In some reviews, explicit causal relationships between certain methods of involvement and positive 
impacts were made. They are summarised in Table 3 below, and ordered according to the research 
cycle outlined in one of the systematic reviews included in the narrative synthesis °°. 


Table 3: Causal relationship between involvement and positive impacts 


Type of involvement Stage of research Impact on research 
cycle 

Identifying topics Agenda Setting Generic positive impact 

Participant recruitment Recruitment and Improved recruitment 
participation 

Developing questionnaires Data Collection Improved relevance 

Interviewers Improved data collection 

Data analysis Data Analysis Improves interpretation 

Influencing dissemination plan | Dissemination Generic positive impact 


Impacts of involvement by stage of research 


The following impacts were recorded at the following stages. Where available, information about 
the type of involvement method used has also been included if it was present in the data line. They 
are summarised in the tables below and ordered according to the research cycle outlined in one of 
the systematic reviews included in the narrative synthesis °°?. Agenda setting and data collection 
were the stages with the most impacts (13), with dissemination the next highest (10). These impacts 
are summarised in Figure 2 and Table 3. 


Figure 2: Number of impacts by stage of research 


m Agenda setting = Study Design and Procedures = Recruitment and participation 
= Data Collection = Data Analysis = Dissemination 
= Implementation a Evaluation 


Table 3: Number of impacts by stage of research 


Stage of research Number of impacts 
Agenda setting 13 
Study Design and Procedures 0 
Recruitment and participation 4 
Data Collection 13 
Data Analysis 2 
Dissemination 10 
Implementation 
Evaluation 1 
All stages 2 
Discussion 


Variation in language 


As this narrative review examines systematic reviews which themselves look at papers from the past 
decade, there is a natural lag in reflecting more contemporary language. 

While ‘user’ still has a very specific and helpful meaning in some contexts (as a shortening of ‘service 
user’), words such as community appear to be more commonly used in more recent literature to 
describe the wider public or community. It is likely that it appears so frequently in this analysis of 
systematic reviews as it is naturally more focussed on past usage of language, rather than reflecting 
contemporary trends, such as a shift to phrases such as ‘community’, ‘public’ or ‘people’. 


Types of involvement 


It is unclear if involvement in data collection and analysis was most reported because this was seen 
as the easiest or most effective way of involving people, or because it was simply the most reported 
method of involvement (with other forms of involvement not being mentioned in papers as it was 
not strictly considered part of the research method). 


Conclusions 


Involving people in any way is generally advantageous, however small it may seem, there may be 
significant and unpredictable positive impacts. Involving people as early as possible is best-practice 
and will likely have significant positive impacts on every stage of the research cycle. While there may 
be some power shifts during this process (researchers sharing control of the agenda, outcome 
measures etc), if planned, resourced and facilitated well, the risk of any potential negative conflicts 
can be effectively reduced. 


Time and budget are oft-cited perceived barriers to involvement. While the longer-term solutions to 
this will require a cultural change by both researchers and the public (to effectively resource the 
important activities required to effectively involve the public) in the short term, there are a number 
of high impact actions which can be taken which do not require considerable budgets or time. These 
include creating steering groups or committees, carrying out surveys, training people to work in 
partnership with communities and using events to raise the profile of the research and the ways that 
people can be involved. 


It is not clear if face to face or online involvement is more effective with regard to impact — and the 
definition of effective will likely change with each individual. Involving people face to face and online 
both have risks of excluding people. However, while the value of face to face events is clear from a 
number of reviews — the ratio of cost to impact is much higher for online involvement. The 
increasing number of people with access to online tools will mean that it is likely that in the future, 
the default action for involving people will initially be engagement with online communities in order 
to consult them about the most appropriate next steps to plan involvement. 


The final principle is that involving people in planning how the public will be involved is crucial. Just 
as with the research itself, if people are not involved in planning how people will be involved, doing 
it and measuring the impact, there is a danger that the method will be ineffective and the value and 
impact of the research will be reduced. 


Applying the principles of involvement to genomics 


The narrative review, combined with the international mapping of genomic projects suggests 
genomic research should not be treated with exceptionalism in relation to strategies for involving 
people. In other words, it is unlikely that involving people needs to be approached in a radically 
different way when carrying out genomic research. 


Some of the most important issues in genomic research are not unique to genomic research. For 
example, how are communities of interest defined; how do the principles of fairness and equality 
interact with potential conflicts of interest when prioritizing research; and privacy and data sharing 
concerns. However, with the number of people accessing genomic testing increasing, with direct-to- 
consumer testing growing every year, it is inevitable that even more people will begin to use online 
tools to both interpret their genomes, share their data and create communities of interest based on 
shared variations (including familial connections or variations of known significance). As a result, 
both the narrative review and mapping suggest that involving people in genomic research online will 
likely have a greater impact in improving research. Face to face events will be less effective as the 
diversity, complexity of communities of interest and geographical distribution inevitably means that 
the cost of such events would preclude their effectiveness. In conclusion, standard models for 
involving people online in every stage of genomic research are required, including standardised 
frameworks for both evaluation and impact assessment. 
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Sharing the 'who', 'how' and 'what' 


Public consultation report 
September 2019 to May 2021 


About this document 


This document describes how the public were invited to be involved in giving feedback on the 
‘Standardised Data on Initiatives (STARDIT): Alpha Version’! between September 24" 2019 to the 
end of 2019. The feedback from this process was summarised into learning points and actions which 
were used to inform the co-creation of the Beta version of STARDIT“?2. This report then describes the 
public consultation process for ‘Standardised Data on Initiatives (STARDIT): Beta Version’ from 
February 2021 to May 2021. 


This report is licensed under a Creative Commons Attribution-ShareAlike 4.0 International Licence. 
This report has been written by Jack Nunn, Director of Science for All and PhD researcher at La Trobe 
University. This project is being run in partnership with the Wikipedia Journals (Wikimedia 
Foundation). More information can be found at ScienceForAll.World/STARDIT 

This report is available in the public domain.7*? 


https://doi.org/10.26181/611dfcf12c6a9 


This report has been included in the appendices of this thesis to indicate the detailed co-design 
process of STARDIT, which occurred in parallel with this doctoral research. 


Public consultation 


Consultation period: September 2019 to December 2019 


The ‘Standardised Data on Initiatives (STARDIT): Alpha Version’(1) was published in September 2019. 
Opportunities to be involved in co-creating this version were advertised online using social media 
and shared via email to potential authors. Comments from co-authors were then incorporated into a 
series of versions, with all co-authors reviewing the final pre-print version. 
The pre-print was shared online and promoted using emails, newsletter and social media. Feedback 
from the public gathered by: 

e = Emails 

e Phone and video calls 

e Online discussion forums 

e Online forms 

e Public events in London and Melbourne 

e Face-to-face conversations 


More information about the public event in London is shared in the next section. Relevant learning 
points from a Wikimedia Youth Salon is also incorporated into this report. 


In addition, Jack Nunn (Director of Science for All) worked with a number of people to complete 
STARDIT reports, in order to test how appropriate and useful the data entry was. This involved a 
series of phone and video calls, followed by exchanging versions of STARDIT reports in order to 
create finalised versions. 


Data from all these sources has been collated and organised into themes using qualitative thematic 
analysis. Event attendees were invited to ensure this report captured comments from the event. 
Further information about this data (including how it was collated and analysed) will be shared in 
the planned peer-reviewed paper ‘Standardised Data on Initiatives (STARDIT): Beta Version’. 


London Event Summary 


On 1% October 2019, Science for All facilitated the first public meeting about ‘Standardised Data on 
Initiatives (STARDIT)’. The event was facilitated by Jack Nunn (Director, Science for All) and hosted by 
the University College London Institute of Education, London. 


Registration was free and open to anyone. People could join both in person and online. The 
facilitated discussion lasted three hours, with breaks. A detailed facilitation plan can be found in the 
supplementary materials. 


Learning from the discussion has been incorporated into feedback from other sources and has not 
been attributed to individuals. 


List of attendees 


In person: 


Online: 


Jack Nunn — Director, Science for All, Strategy Liaison and Editor for the WikiJournals, 
member of the Cochrane Advocacy Advisory Group, PhD candidate at La Trobe University, 
Melbourne (Australia) 

Sandy Oliver - Director of the Social Science Research Unit and Deputy Director of the EPPI- 
Centre, Professor of Public Policy at University College London, Editor of the journal 
‘Research for All’ 

Carolyn Thompson - PhD Researcher, Institute of Zoology and University College London, 
Postgraduate Teaching Assistant and Lecturer, University College London. 

Mick Mullane - Innovation Lead, National Institute for Health Research Digital Office 

Jim Elliot - Public Involvement Lead, Health Research Authority (England) 

Richard Stephens - Patient Advocate, Co-Editor-in-Chief, ‘Research Involvement and 
Engagement’, National Cancer Research Institute ‘consumer’ representative 


Chloe Mayeur — Sciensano (Belgium) 
Wannes Van Hoof — Sciensano (Belgium) 
James Ansell — Consumers Health Forum (Australia) 


* ` _ 


The first ‘STARDIT’ selfie at the London event 


Left to right: Jack Nunn, Sandy Oliver, Carolyn Thompson, Mick Mullane, Jim Elliot, Richard Stephens 


Consultation period February 2021 to May 2021 


After the feedback from the Alpha version was collated, work began on the Beta version. Between 
January 2020 and February 2021 multiple meetings took place (with some face to face involvement 
cancelled owing to the COVID-19 pandemic). 


Online video presentations and discussions 


Online activities where feedback on STARDIT was invited and given include (but are not limited to) 
the following invited presentations and discussions: 


Title Invited by Given by Date 
Standardised Data on WikiCite 2020 Virtual conference | Jack Nunn and 27" 
Initiatives (STARDIT)4%4 Thomas Shafee | October 
(recording, 2020 
resource and 
transcript) 
Standardised Data on Poche Centre for Indigenous Jack Nunn 18" 
Initiatives (STARDIT)*%> | Health, 9th Annual Research November 
Showcase Program 2020 
Involving people in Poche Centre for Indigenous Jack Nunn 8th 
DNA research Health, Research Advisory Board September 
2020 

Involving People In Ludwig Boltzmann Gesellschaft Jack Nunn 24" 

DNA Research (recording, September 
resource, 2020 
presentation) 

Genomics Research La Trobe University Jack Nunn 13% 

and Involving People*®? October 

2020 

Involving everyone in Australian Citizen Science Jack Nunn 1* April 

research: Creating the | Association 2021 

evidence*® 

Involving People in Rare Voices Australia Jack Nunn 5th August 

Rare Disease Research 2021 


Text-based feedback and discussion 


Method 


A number of methods for gathering feedback and hosting online text-based discussions were used 
for this period of the consultation. This included using an online text-based discussion platform 
(hosted pro-bono by Science for All) to discuss the STARDIT Beta version, online forms for collecting 
feedback, online shared documents for simultaneous editing and commenting and using online pre- 
print servers to share stable versions**. A version of the Science for All STARDIT Beta webpage was 
archived to preserve how feedback was invited during this period™?. Information about the 
consultation process was also shared by STARDIT authors via email, social media (including Twitter, 
Facebook and LinkedIn), and the Science for All website. Specific areas where feedback was 
requested included: 

e Helping improve areas which are unclear or might not make sense 

e Checking the STARDIT data fields are appropriate (anything missing or unclear) 

e Improving the ‘Example applications of STARDIT’ table for your own discipline(s) 

e Suggesting any relevant references that may be missing 


Results 


A total of 27 people provided feedback on the Beta version via the online form and collaborative 
document. Over 7000 words of feedback and comments were provided via the online form with 144 
separate points, comments or corrections. While there were multiple small changes and comments 
on the collaborative document, there were 51 comments with logged changes which were 577 
words in total. 

All of the feedback, comments, corrections and responses by the lead authors can be found in the 
supplementary resources, in the section ‘Anonymised Feedback on Beta Version’ and ‘Change log 


from Alpha Version’. 
The final version was sent to all co-authors for checking before submitting for publication. 


Learning points from the public consultations 
STARDIT as a project 


e The principle of standardised reporting described by STARDIT is useful across disciplines, 
this is ‘unique’ 

e STARDIT reports will be useful for a number of disciplines, including health research, 
environmental research, public policy, educational interventions and community arts 
projects. 

e Many people don’t know who to trust and one participant noted that ‘most of our decisions 
are based on trust’. STARDIT was identified as a way of sharing data that will facilitate 
people to critically appraise many kinds of data. 

e STARDIT is especially helpful for people to self-assess research and appraise it, including 
supporting informed decision making about whether to participate. 

e  STARDIT was identified as a way of reporting how people were involved in defining ‘shared 
purpose’, including defining outcome measures (for example, answering the question ‘what 
does success look like and how will we measure it’) 

e STARDIT could provide an independent way for researchers and policy makers to show 
how people have been involved in co-producing it 

e STARDIT was identified as a helpful system for international development, including 
planning, reporting and evaluating initiatives” 

e STARDIT was identified as a helpful system for people planning, reporting and evaluating 
initiatives, including mapping preferences for involvement, reporting involvement and 
impacts from involving people. 


Proposed collaborative way of working 


e While the project is ‘ambitious’, the proposed collaborative way of working balances 
openness with efficiency 

e In order to make STARDIT happen, it was suggested to ‘start small’ and ‘think like a start-up’ 

e Anumber of partner organisations were suggested throughout the public consultation 
including Academic Health Science Networks (UK), The National Cancer Research Institute 
(UK), Independent Cancer Patients’ Voice (UK), Clinical Trials Units (UK), Patient Focused 
Medicines Development (global), National Institute for Health Research (England), Good 
Things Foundation (UK and Australia), Google (‘Scholar’ team), National Institute of Health 
(USA), Patient-Centered Outcomes Research Institute (USA) and the James Lind Alliance 


Authenticity and trust 


“This is so global and so big — it comes back to trust, how do I trust the people who report data 
using STARDIT” 


STARDIT will be assessed by an editorial board and eventually, open peer review. It will use 
indicators from public domain sources. However, the root problem of authenticity and truthful 
reporting remains for all peer-reviewed data. While STARDIT provides data to facilitate critical 
appraisal, ongoing work will be required to ensure the authenticity of data. Partnering with the 
Wikidata project will ensure data is linked and machine-readable. Assigning Digital Object Identifiers 
to STARDIT reports will ensure that versions are immutable, but that the reports themselves can be 
updated should errors or inaccuracies be discovered. 


Personal safety risks 


Risks were identified with STARDIT for people who may share information or data which might have 
legal or safety implications. For example, data provided by members of the public about illegal 
activities (such as poaching or illegal logging) might incriminate individuals or put those sharing the 
data at personal risk. Ways of ensuring data is shared in ways which balance transparency with 
personal safety need to be carefully considered. China was identified by one researcher as an 
example of a country where special attention and cultural sensitivity would be required. 


Life or death information 


STARDIT was identified as a system which could help people critically appraise information which 
might be life-saving or potentially lethal if incorrect. As well as medical information, this also 
includes information on Wikipedia pages about things such as edible fungi and plants°”’. 

STARDIT should have a transparent process for redacting information which might contribute to the 
destruction, poaching or killing of rare or threatened species, for example, not sharing detailed 
location information of rare species. 


Sharing power 


There is ‘knowledge as power and powerful knowledge’, STARDIT is a way of sharing both kinds of 
knowledge. Some ‘power brokers’ might not welcome knowledge sharing, transparency and scrutiny 
in certain areas and may actively resist attempts to share data and power. ‘Power brokers’ who 
might be resistant were identified as people working in government and for-profit organisations. 


Knowledge translation 


Understanding and measuring comprehension and knowledge translation are ongoing challenges in 
many disciplines. While STARDIT can report data on this, ongoing work will be required to ensure 
reporting is aligned with international best-practice. Partnering with organisations such as Cochrane 
and Campbell will help ensure the reporting tool is useful. STARDIT can report transformative 
learning as an impact, but this will require careful tailoring to each language and culture. 


Diversity and inclusion 


Ensuring the process for both involving people in the development of and for using STARDIT are 
inclusive will need continuous reassessment, potentially requiring a group of experts and advisors. In 
addition, as STARDIT is developed for languages other than English, groups of people specialising in 
linguistic and cultural diversity will need to be involved in ensuring STARDIT is appropriate, culturally 
safe and inclusive. In addition, learning and development opportunities will need to be co-created 
with multiple stakeholders in order to ensure people are given inclusive opportunities to learn how 
to get involved with the STARDIT project. This was raised as a particular consideration of Indigenous 
peoples during one presentation to the Poche Centre for Indigenous Health. 


Technical considerations 


Machine learning and ‘artificial intelligence’ could be employed to create reports. Wikidata is built 
for machine learning and provides an open, public domain and free way of sharing data that anyone 
can access, anywhere. After providing a way to host reports, multiple ways to submit them should 
be co-created. 


Readability and plain English 


The ‘Standardised Data on Initiatives (STARDIT): Alpha Version’ needs to be improved for readability 
and plain English. In addition, the purpose and scope needs to be explained more clearly. Tailoring 
communication to specific disciplines should also be considered. Future versions that might be 
translated into other languages will require co-creation with language communities to ensure they 
are comprehensible to as wide an audience as possible. 


Systematic Searching 


Future versions of STARDIT after the Beta version will require a systematic review in order to ensure 
that all appropriate data sources have been consulted. As this will require a significant investment of 
time from those involved in the STARDIT project, it was agreed that at this stage of the co-creation 
process, a ‘mini-review’ (published in the peer-reviewed WikiJournal of Science) was an appropriate 
intermediate step to ensure the current search strategy is appropriate. 


Indigenous knowledge 


A report by Science for All written for the Wikimedia Foundation identified that there might be 
systematic, technical and cultural barriers to incorporating the knowledge of indigenous peoples into 
Wikipedia and other peer-reviewed repositories”. After additional meetings with staff from the 
Wurundjeri Woi Wurrung Cultural Heritage Aboriginal Corporation, it was agreed that it could be 
helpful to explore using STARDIT to co-create a way for indigenous peoples to share traditional and 
oral knowledge. STARDIT could be used to transparently report who created any content containing 
the knowledge, what tasks they had, how this knowledge was shared and any relevant concepts of 
‘owning’ or ‘property’. Members of Indigenous communities could work in partnership with the 
Wikimedia Foundation to create ‘verified’ users who formally represent relevant communities are 
have permission to share and verify knowledge (including stories, beliefs, medicine). 

The report concluded that a detailed piece of research needs to be commissioned (potentially by the 
Wikimedia Foundation) to explore concepts of ‘intellectual property’ and ‘owning knowledge’, and 
how this respectfully interacts with the free knowledge and open access movements. Certain 
cultures have restricted, taboo or ‘secret’ knowledge®>’. This can include culturally significant sites 
which may be at greater risk of vandalism if they are shared in the public domain. STARDIT needs to 
be co-developed with the Wikimedia Foundation and with indigenous peoples to ensure that a 
balance is struck between sharing, storing and preserving unique intangible culture, while also 
remaining sensitive to respective cultural practices and attitudes regarding ‘ownership’. 


Actions and results 


1. 


Create a peer-reviewed scoping review to supplement the STARDIT beta paper, in preparation 
for a future systematic review. 


Result: Completed and ready for submission 


Science for All to pay developers for creation of STARDIT report hosting. Create a project brief 
and invite developers to apply to create a beta version of STARDIT hosting. 


Result: Completed, Beta version built by paid developers (paid by Science for All, approved by 
the Steering Committee) 


Host more face to face and online events in other capital cities, including Canberra and Berlin. 


Result: Abandoned. Planned face to face events in London, Berlin and other cities in 2020 were 
converted to online meetings and presentations 


Rewrite parts of the STARDIT paper to be clearer and in plain English. 


Result: Completed , Beta version ready for submission in open access journal 


Supplementary resources 
Facilitation plan for public event — 1% October: London 


Purpose of event 


Create an opportunity for people from different disciplines to talk about standardised ways of reporting initiatives, including research, education and 
international development. 


Aims 


e Summarise what different disciplines are reporting about initiatives and how 
e Suggest a common framework for reporting (STARDIT) 
e Host a discussion about common challenges and generate ideas 


Introductions A chance to learn who is in the room, and what they Ask everyone in room and online Everyone knows 
hope to get out of today - and what the process for the | to say what their area of expertise | who is in the 
afternoon is or knowledge is and why they’ve room and online. 

come today - Online: Facilitator 
will summarise comments from 
people 


Jack Nunn, A short presentation from Jack Nunn, Director of 

Director of Science for All - about the learning from his recent 

Science for All projects, including his PhD about involving people in 
genomics research. A short summary of ‘Science for 
All’ and what led to STARDIT 


Carolyn A primatologist investigating small ape decline in 

Thompson, China, Vietnam and Myanmar. She’s working with local 

Primatologist people, using participatory action research to 
investigate the patterns and drivers of critically 
endangered gibbon decline. She will discuss how 


STARDIT could be useful for recording impacts from 
this kind of work. 


Open discussion (including input via Zoom) Ask people online to type thoughts | People online and 
or comments to be read out (also | in the room have 
check Twitter #stardit). contributed to 
Ask people in the room to share discussion 
initial thoughts, summarise 
comments from people online. 


Break - mingle - tell people online to log back on in either ten minutes to hear speakers or log back on at 2:10 to join the ‘Idea 
Vortex’ - note Australians might be going to bed and say goodnight 


Reporting the Short introduction to STARDIT with 3 different 
what, who and speakers leading discussions on how it could be used 
how? and improved. 


Sandy Oliver - a personal and professional perspective 


(including thoughts on journals) - lessons from 
successful reporting tools? 


Richard Stevens - a personal perspective as a cancer 
patient and a professional perspective in relation to 
genomics research and on journals) 

What would make people want to use this? 


Jim Elliot - a personal perspective and a professional 
perspective in relation to the work of the Health 
Research Authority. What support would people need 
to use STARDIT? 


Idea vortex Using the ‘Idea Vortex’ model - a series of questions Welcome back people online! 
designed to find issues and create solutions 


Learning so far A chance for anyone to speak about what has been Invite comments from people 
learned so far, any reflections. Also a chance to map online 
who’s not involved who should be moving forward - 
and who will invite them! 


Agreeing tasks, What actions have been agreed, what tasks and areas | Explain how Loomio will be used A list of actions, 
actions and for further discussion going forward and how actions tasks and 
discussion areas and decisions will be made decisions to be 


posted on 
Open discussion - tea - cake - ‘networking’ 


Loomio. 


Formally close event 4pm. Adjourn to nearby pub. 


Additional Discussion Points 


These additional discussion points were used to supplement the discussion: 

e Can anyone write STARDIT reports? People unaffiliated with projects? Can this be one data-line 
that contributes to a ‘living report’ - in other words, could people report on behalf of 
organisations (like people can write Wikipedia pages about organisations without their approval) 

o Solution could be that reports make it clear when people from the organisation have 
been involved - verified (tick like on twitter?) 

e How should peer review work? 

o Inthe short term, it will have to be an editorial board (volunteers associated with the 
WikiJournals) - we will use the existing processes of the WikiJournals for the Alpha 
version and Beta version 

o Inthe longer term - post beta version - it should be an open peer review process. For 
discussion 

o Peer review needs to ask the question is there evidence/data to back claims in STARDIT 
report - does it require some kind of standard critical appraisal tool? 

e Inthe longer term STARDIT could ‘score’ projects 

o STARDIT scored- a peer reviewed score for an initiative which scores it on criteria 
including ‘power sharing/involvement’, data sharing, dissemination and translation 

o Scoring could be based on ‘is there a data source for this item’ so that it is not subjective 
(Binary yes or no on indicators of involvement) 

o Scoring continually reviewed but must be future-proofed so historical scores still have 
validity and use 

o This function would likely require funding/grant etc to support infrastructure - while peer 
reviewing would be voluntary the process of editorial control and back end would need 
not-for-profit investment. 


AO o 


e Can things like ‘documentaries’ be included - who made it, who did what, who funded it? Would 
this be a category of ‘educational intervention’ - allowing documentaries to actually measure 
impact 

e Risk of confusion with reporting guideline: STARD http://www.equator-network.org/reporting- 
guidelines/stard/ 

o Not considered an issue by attendees 

e Create API for other journals etc to use with their site? Create badge 

e Partners get accredited to improve participation and recruitment 

e Citing Aboriginal stories - create STARDIT report for story? 


Respondee ID 
N 


Anonymised Feedback on Beta Version 


Feedback with an action for the paper 


Need examples earlier on exactly WHAT you are collecting - 


The challenge for any non-research person reading the abstract and the lengthy 
preambles in the paper is always, "Yes, but WHAT data?" There are no examples 
given until page 10 of the main paper, where there are some very useful and 
comprehensive lists. Even then as a lay person | think of it as "information" not 
"data". All the more reason why you need examples much earlier on of exactly 
WHAT you are collecting - you mention responding to an epidemic, which is a great 
outcome, but STARDIT is about process, and that's what isn't clear. 


General feedback about STARDIT 


mention of patients/public being able to complete a STARDIT report themselves, but would 
research teams or those responsible for patient/public activities need to deliver training to 
equip them with the skills to complete the form or do you envisage the reports being a joint 
effort? 


132 & 133 


mention responding to an epidemic, which is a great outcome, but STARDIT is about process, 
and that's what isn't clear. 


Change made 


added to discussion: 'Similarly, based on feedback 
from Indigenous community leaders, patient 
representatives and others, it is essential to ensure 
access to learning and development opportunities is 
available to support people to both access and create 
STARDIT reports.’ 


See log from tracked changes 


Very helpful point, thank you. | have added this to 
background section 'For example, when designing a 
response to an epidemic, standardised data can 
improve retrieval of relevant information which can 
be used to inform which affected individuals or 
organisations could be involved in the design of the 
response and which outcomes are most important. 
This can include deciding which stakeholders should 
be involved in which tasks, such as prioritising 
outcomes." 


| also added this para to differentiate between raw 
data and metadata, as both of those are included in 
what we refer to as ‘data’ - 'Hereafter, data 
generated by an initiative (including raw data), 
information about the data (meta-data) and 
information about the initiative will all be referred to 
as ‘data’ unless otherwise specified." 


Lead author's response if no 
change made 


Second Author Response 


Status 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Respondee ID 


Feedback with an action for the paper 


line 405, Table 4: first, regarding the Section “Involvement in initiative”, Data 
category “Involvement appraisal”, data field “How did the initiative change as a 
result of involving people (did the design or evaluation change?)”. Suggest not only 
ask for effects of involvement on the initiative where involvement took place per 
se, but also on other areas. 


General feedback about STARDIT 


If these categories are supposed to be described under the data category “Involvement 
outcomes, impacts, learnings or outputs” | would be a bit more specific, because right now, the 
data field description “Where there any outcomes, impacts, learnings or outputs from people 
begin involved?” reads to me as if this category focuses solely on effects on the people — which | 
think does deserve its own category — but not on what the people (and their organizations) may 
do based on their involvement. | do think this aspects differs from the section “Impacts and 
outcomes” because it may be directly correlated to the involvement of the project. | am 
thinking, for example, of a research project that involved local practicioners. One of them told 
us that he started applying the participatory methods he experienced in the research project as 
a participant in his work environment at a social service organization. | would not classify this as 
an outcome from the project per se, but from the involvement directly. It is, however, “more” 
than just an outcome on the people involved, but a “spill-over” from their involvement to affect 
others outside the involvement process. 


Change made 


Thank you for this, | think all the things you identified 
in your comment can currently be covered by this 
point, however | agree in future versions perhaps this 
could be articulated further. | have added to the help 
text ‘These can include impacts on people, 
organisations, processes or other kinds of impacts' 
which | hope addresses your point? | will also flag 
with Thomas whether this section should just be 
added to the other impacts categories, although | 
note this is more of a design interface issue rather 
than a structured data issue. 


Lead author's response if no 
change made 


Second Author Response 


| think it should be possible to create structured data 
to indicate these sorts of outcomers something like 
this (e.g. using "applies to part (P518)" to indicate the 
person and “caused by (P828)"+"involvement 
(Q1671829)" to indicate that it waws the act of 
involvement that lead to said outcome). Indeed 
indicating if any outcomes apply to a specific 
person/group may be useful in other circumstances 
(e.g. applies to study participants, some 
governmental department, NGO, entire industry 
sector, etc) 


Having said that, | don't know whether it's going to 
be common enough that we'd want a whole separate 
section for it in terms of a data entry form. I'd be 
keen to have it simply as an option in the normal 
outputs section. 


Status 


Comment 
incorporated 
into Beta 


Feedback with an action for the paper General feedback about STARDIT 


line 405, Table 4: section “Impacts and outcomes”. | am wondering whether a 
section about who decides on the outcomes used to evaluate the impact of an 
initiative may be helpful. 


| think it would be quite interesting to know if the outcome measures were decided on 
that often those initiating the project (in our case, researchers) have quite different goals and 
only initiators of the project decide on outcome measures, these measures may not reflect the 


that reflects the interests of only one party rather than all. Therefore 


n 
© 


Note- | answered ‘no’ on the first two questions even though the technical answer 
to the question(s) is (were) ‘yes’ but both the Abstract and Plain-English need a but 
of tweaking/clarification. 


© 


Line 38: suggest ‘aims’ instead of ‘exists’ potentially a better fit 


Line 44-45: noting open-access is repeating info given in line 39+40 so is redundant. 
Unsure why it is being noted here that ‘authors can be verified’, suggest delete or 
elaborate on why this is important. 


© 


Line 44-45: Data being “assessed for quality” is not plain English, needs refinement 
and clarification. 


Line 46: suggest ‘counter complex global problems’ rather than ‘improve’ 


Line 47: delete “being” 


e 
© 


Line 97+98: adding a third example in the ‘pandemics and air pollution’ in line 97 
(ie. pandemics, air pollution and X’) 


collaboratively as well or who was in charge/what was the process of agreeing on them. I found 
outcomes in mind than participants — sometimes these goals can even contradict themselves. If 


full reality of what the project really “should” lead to, but only a limited (maybe distorted) view 


Change made Lead author's response if no Second Author Response Status 
change made 
Thank you - after careful thought | agree and decided Comment 
this did need extra categories. While strictly this incorporated 
whole section could be put into methods, | think it's into Beta 
important to make it distinct and clear. As a result | 
added 'Who was or is involved in deciding on the 
outcomes used to evaluate any impacts or 
outcomes? How were they involved?’ - | note this 
could be two data categories but for now kept them 
as one row in the table just more to keep table 
length down than anything. 
Please share any further changes you think should be comment 
made noted but not 
incorporated 
into this 
version 
Thank you - | went with 'was created' as it's more Comment 
active and concrete incorporated 
into Beta 
changed to 'STARDIT is free to use and data can be Comment 
accessed or submitted by anyone. The authors of the incorporated 
data can be verified (to improve trust)’ into Beta 
changed to 'checked' Comment 
incorporated 
into Beta 
changed to 'solve' as counter is not plain english Comment 
incorporated 
into Beta 
changed Comment 
incorporated 
into Beta 
added ‘biodiversity destruction’ although climate Comment 
change could be another example to use, sadly too incorporated 
many to choose from into Beta 


e 
© 


n 
© 


e 
© 


© 


© 


© 


© 


Feedback with an action for the paper 


Line 102: putting ‘research’ into quote marks like the terms in line Line 103. Just for 
consistency and to emphasises all the terms are equally valid. 


Line 49: “interventions in which affected population groups are integrally involved” 
is a bit convoluted. Needs tweaking/shortening. 


Line 69-73: not clear as written how exactly Stardit as a mechanism leads to the 
benefits explained. Elaborating that it does these things through making retrieval 
and comparison of data easier/more efficient/faster to lead to better and quicker 
decisions. 


Line 34-73: clarifying the distinction in these two sections (and in the Beta as a 
whole) between Stardit as a tool/concept and the Stardit repository of information. 


Line 118-120: the situation being highlighted here is bad but should tease out why 
exactly to lay groundwork for how Stardit will solve it. 


Line 115-120: This paragraph is a bit confusing and the three 
statements/information bits don’t seem strongly linked with each other. Starts by 
talking about the importance of involving broad groups in initiatives generally but 
then ends talking about a problem of too many reporting tools in a specific type of 
data. 


Line 127: unsure if ‘compete’ is the right word. Do we mean ‘conflict’? 


Line 202: should ‘Participatory Action Research’ be capitalised? 


Line 208: Critical reflexifivity should maybe be in quote marks and/or be capitalised 
given it’s the name of a, uhhh, theory(?) (methodology?) 


Line 215-216: unsure what ‘media freedom’ means in this context and how it is 
relevant to Stardit. Presume that it is referring to a free media not controlled by the 
State but that doesn’t really seem to clearly link to Stardit processes or aims. 


General feedback about STARDIT 


Change made 


changed thank you, good spot 


Changed to 'STARDIT was developed on the 
understanding that such problems require evidence- 
informed collaborative methods, multidisciplinary 
research and interventions in which people who are 
affected are involved in every stage.' 


Changed to 'For example, when designing a response 
to an epidemic, standardised data can improve 
retrieval of relevant information which can be used 
to inform which affected individuals or organisations 
could be involved in the design of the response and 
which outcomes are most important.’ 


| have worked to address this throughout but it's a 
valid point (and a nuanced one that's challenging to 
explain in plain english) 


added 'In addition to providing new standardised 
data categories for describing who was involved in 
which tasks, STARDIT can also incorporate existing 
data standards (see the supplementary resources 
‘Using Standardised Data on Initiatives (STARDIT): 
Beta Version Manual’), creating a unifying system for 
data hosting, linking and analysis. ' 


added ‘compete or conflict' as these are distinct and 
important 


changed to 'STARDIT development is guided by 
participatory action research (PAR) paradigms' as it's 
not a proper noun and the singula use implied 
there's only one paradigm 

whatever it is it's certainly a jargon term (but one 
that can't be avoided, as that's what it's called!) so 
I've changed to ‘At the core of participatory research 
is ‘critical reflexivity’ 


changed to 'To uphold human rights and 
‘environmental rights’64, and for ‘the maintenance 
of peace’, people require ‘media freedom’ in order to 
‘seek, receive and impart information’ 63, free of 
unaccountable censorship! - | would say media 
freedom and censorship are likely to be at the very 
core of some of the reasons STARDIT might be 
challenged by some people with power to challenge 
such things 


Lead author's response if no 
change made 


Second Author Response 


Status 


Comment 
incorporated 
into Beta 
Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Feedback with an action for the paper 


e 
© 


Line 216-217: How will/does Stardit held to ‘uphold these universal rights’? 


e 
© 


Line 281: there is an errant * against ‘Genomic research’ that does seem to 
correspond to any post-table notes 


Line 292: unclear where people are meant to start in this Figure i.e. which is step 1. 
adding numbers like in line 303. the line ‘Report planned initiative’ have the 
addendum ‘into Stardit’ into it. Similailrly ‘preference mapping’ should maybe be 
standardised/expanded to be ‘Stakeholder preference mapping’ as currently 


H 
© 


Line 303+305: Ditto prior comment about saying ‘Stakeholder preference mapping’ 
consistently for clarity 


n 


Line 317: citation needed for the study being referred to 


© 


Line 326: Table X needs to be updated with relevant number 


e 
© 


Line 346: In the fourth row of the table, the second column should maybe read “To 
establish the purpose, motivations and values of the research from different 
viewpoints” or similar 


n 


Line 406: Maybe title should read ‘Discussion and Next Steps’? 


Page 75- errant highlighted text Page 83- errant track change spell check to be 
approved 


e 
© 


Page 160- in the bottom row of the table, righthand column, on the second last line 
it should read ‘there may be formal’ not ‘the may be formal’ 


n 
© 


Page 105, line 164- another ‘Table X’ needs number inserted 


General feedback about STARDIT 


it isn't clear what the term means within the figure. 


Change made 


Thank you yes that was missing - added 'STARDIT has 
been created in order to help anyone uphold these 
universal rights, by providing a way to share open 
access information in a structured way with a 
transparent process for quality checking’ 

removed 


Added 'Figure 2 describes how STARDIT can be used 
to map how people might be involved in designing, 
doing, reporting and evaluating initiatives, starting 
with ‘idea sharing’, to clarify the cycle, but | guess the 
point is you can start at any stage of the cycle 


RE Pref mapping, reworded to this for clarity ‘The 
STARDIT Preference Mapping (STARDIT-PM) tool 
provides a standardised way to report the preference 
of multiple stakeholders.’ | think adding the word 
‘stakeholder' is redundant, as who else would be 
having preferences mapped? Will check with Thomas 
on this one 


Thank you good spot 


agreed - changed to 'To establish the purpose of the 
research, and the motivations and values of the 
initiative from multiple perspectives’ 


changed to ‘Discussion and future versions' to reflect 
the content 


eagle-eyed! changed 


Incredible spot, you really did read it all! Thank you, 
corrected 


Changed! 


Lead author's response if no 
change made 


Second Author Response 


It may be worth including the redundant 
‘stakeholder’ just to emphasise that it is for 
stakeholders generally (or indeed a subset of 
stakeholders) as readers may have implicit 
assumptions that preference mapping might just be 
for a specific group (e.g. investors) or something like 
that. 


Status 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 
Comment 
incorporated 
into Beta 
Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


m 
is) 


is) 


m 
co 


Feedback with an action for the paper 


Line 281: Table 1: Example applications of STARDIT. involve different parties is the 
evaluation of marketing authorisations, and also appraisal. Health Technology 
assessment is mentionned, but not medicines or medical devices regulation 
(authorisation) and/or appraisal/pricing. 


Line 403 - Table 4: Summary of STARDIT Beta Version data fields 

In data, there are different methods used to anonymise data so that individual 
cannot be re-identified. 

Line 403 - Table 4: Summary of STARDIT Beta Version data fields 

There is a varying risk of re-identification (from 0.5 to 0.05 or less, depending on 
which anonymization method is used and of the context). 


LINE 63: Make this far more generic and inviting to anyone engaged in activities 
that have potential to inform science. Talk about the range of activities later, with 
broad descriptions of each, so that those unfamiliar with any or all terms can look 
them up and acknowledge. 


General feedback about STARDIT 


It would be important to mention 1/ the risk of re-identification, 2/ the method used to de- 
identify data 


It would be important to mention 1/ the risk of re-identification, 2/ the method used to de- 
identify data 


line 346: financial relationships and other interests are mentioned; however other interests are 
not too detailed. | think they're key, sometimes more important to know that financial interests. 
You might be involved in the early phase of an exciting project, and continune until the end. 
When final results are disappointing, you might not be totally objective when intepreting them 
(participatory type conflict of interest. Other types are carrier interests, intellectual interests, 
conflicts between persons etc.) . Instead of simply "other interests", create a list of all types we 
can think of, otherwise people will not realise they should declare some. 


ABSTRACT 

Given how important framing at the onset is for ensuring the initiative is inviting for people 
(feeling like they can relate to it) and setting expectations, as | find abstracts above 250-300 
words and with subsections quite rare across the 3 disciplines | work in, and some of the 
information seems pretty detailed for an abstract. It’s also good to end an abstract with a 
conclusion-focused line about contributions, impacts, and/or future directions, based on what 
results show to date. | thought it equally worthwhile for me to consolidate a list of initiatives, 
people, and resources | thought you would be interested in, as well as help you to make contact 
with some of those folks too. | hope you find the below comments helpful for the paper more 
broadly as well. 


Change made 


added ‘regulation and authorisation processes (for 
example medicines and medical devices)' to 
‘Production, consumerism and business ' sub 
category ‘Other products (medical devices, 
electronics)'. | also added ‘code and algorithm 
checking (for example, autonomous vehicles)' as | 
think this fits here 

Super important points, thank you. Reworded to 
‘Who controls access to the data, how are decisions 
about data access made? Is data anonymised or de- 
identified? What methods are used for re- 
identification? What is the risk of unauthorised re- 
identification? ' 

Changed to ‘Financial or other interests (including 
personal or professional interests)' and also added 
"Describe any conflicting or competing interests 
(including any relevant information about authors of 
this report), or any other ‘interests’, including 
personal interest or (for example, how you may be 
personally or professionally affected by the outcome 
of the initiative)’. I'm keen to keep ‘interests’ open 
ended as an exhaustive list is impossible, but 
certainly in future versions we can work on 
standardising more types of interests 


reworded abstract to ‘Current reporting methods 
lack information about the ways in which different 
people are involved in initiatives, making it difficult 
to collate and appraise data about the most effective 
ways to involve different people. For example, forms 
of participatory action research where anyone can be 
involved in any aspect of research (including ‘citizen 
science’) are increasingly recognised as crucial 
paradigms for solving global problems, as they can 
help ensure that initiatives are aligned with the 
priorities of those affected, thus redefining what it 
means to be a ‘researcher’.' 


also reworded and re-order background section so it 
doesn't focus on health/citizen science too early but 
uses them as illustrated examples. 


Lead author's response if no 
change made 


Second Author Response 


Status 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 
Comment 
incorporated 
into Beta 
Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Respondee ID 
m 
60 


8 


Feedback with an action for the paper 


cite the following refs with the last point, as they both give an overview of terms 
commonly used for citsci: 

Eitzel, M. V., Cappadonna, J. L., Santos-Lang, C., Duerr, R. E., Virapongse, A., West, 
S. E., Kyba, C. C. M., Bowser, A., Cooper, C. B., Sforzi, A., Metcalfe, A. N., Harris, E. 
S., Thiel, M., Haklay, M., Ponciano, L., Roche, J., Ceccaroni, L., Shilling, F. M., 
Dörler, D., Heigl, F., Kiessling, T., Davis, B. Y., & Jiang, Q. (2017). Citizen Science 
Terminology Matters: Exploring Key Terms. Citizen Science: Theory and Practice., 
2(1). https://doi.org/10.5334/cstp.113 

Kullenberg, C., & Kasperowski, D. (2016). What is citizen science? — A 
scientometric meta-analysis. PLoS One, 11(1), e0147152. 
https://doi.org/10.1371/journal.pone.0147152I. 


LINE 66: I am not sure why researcher is in brackets, it might be more fruitful to talk 
about different forms of knowledge that everyone has, which could make valuable 
contributions to scientific research people’s knowledge. 


LINE 71: [food for thought] The epidemic example seems very specific while also 
vague. Perhaps also adding a biodiversity or social science example could help 
people invasion the broad scope of this effort and how standardisation of all 
projects could allow for innovative cross pollination? 


General feedback about STARDIT 


In my experience, involved in activities that inform science may not feel comfortable with being 
labelled as a researcher or a scientist, but are happy to share their knowledge to help us | know 
more collectively. Professional researchers likewise may feel threatened by language that may 
be perceived as diminishing their work too, and framing around diverse forms of knowledge 
(e.g. experiential and cosmopolitan [vetter]) Vetter, J. (2011). Introduction: Lay Participation in 
the History of Scientific Observation. Science in Context, 24(2), 127-141. 
https://doi.org/10.1017/S0269889711000032 


If wanting a medical example, projects looking at spread of mosquito-borne diseases can include 
medical, biodiversity (e.g. mozzies spread animal diseases like avian flu too), habitat 
assessment, and social impacts (e.g. who has access to different forms of preventative tools and 
meds). 


Change made 


Changed abstract to read ‘blurring the lines between 
concepts such as ‘researcher’, ‘public’, ‘patient’ and 
“citizen” 


Thank you - at the moment | thought to keep it one 
worked example throughout and changed it from air 
pollution to a pandemic, as most people naturally 
have a better understanding of that now - happy to 
use more examples throughout but for now have 
kept as it is as | worry it's too confusing to have too 
many worked examples too soon? 


Lead author's response if no 
change made 


Second Author Response 


Status 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Feedback with an action for the paper General feedback about STARDIT Change made Lead author's response if no Second Author Response Status 
change made 


m 
ic) 


LINE 78: “co-created in a collaborative way” is redundant as “co” created is LINE 78: “co-created in a collaborative way” is redundant as “co” created is cooperative, by reworded: STARDIT has been co-created, involving Comment 
cooperative, by definition, but what you actually mean by “co-design” is the key definition, but what you actually mean by “co-design” is the key part, as people is the term in collaboration with people from around the world in incorporated 
part, VERY broadly and often ambiguously. multiple ways. Informed by a number of literature into Beta 

reviews and guidelines, methods of involving people 

have included public events, online discussions and a 

consultation process. 


m 
0 


TABLE 1: | find it interesting that there is “environmental research” but there is no This implies all projects on environmental must be research, but at a community level, this is I agreed with these two comments. | have created Comment 
mention of other activities that aren’t research based necessarily (e.g. natural often not the case. | also find it odd that this is under the initiative “science for all” and yet the category in table 1 called 'Management and incorporated 
resource management, conservation) . It might be worth acknowledgement science isn’t explicitly discussed much, or a key component of Table 1. | also feel it would be monitoring’ where | include both environmental and into Beta 
explicitly that are diverse ways of gaining knowledge beyond reductionist scientific | useful to This table seems pretty health dominant, which surprised me a bit. natural resource management, and alsp public and 
methods? private essential services (e.g. water/power blur that 

line between natural resource and essential critical 

infrastructure) - but how both of these are ‘managed’ 

and monitored (or not managed and monitored) is 

important data to have. | also included data 

management and monitoring too - but note there's 

already a section about data and code etc. 


have an area explicitly talking about technology, particularly in the digital age of it Comment 
transforming how we gain knowledge beyond traditional conventions of science incorporated 
and the scientific method (e.g. e-science involves data mining, which is very into Beta 
different)? 


FIGURE 2: This figure is interesting to me, and | would really like to know more As a designer myself, | feel like it’s missing some key aspects around understanding the Thank you for this comment - reference added. | feel Comment 
about what you mean by each of the variables included, as many of those could be | practices and cultures of people, their actions in a particular context, and their use of that the design stage is covered in more detail in incorporated 
interpreted in a variety of ways that may not have been intent. These aspects also technologies, with interactive design as new practices develop. Without considering existing figure 3, figure 2 being the standard ‘plan, do, into Beta 
apply to Figure 3. conditions, constraints, variables, etc in a local context, projects often fall down. The paper evaluate’ cycle, figure 3 being the ‘design’ cycle 

exemplifies exploring peoples practices and actions to understand feasibility and needs for which incorporates those elements you've 

future projects: Oliver, J. L., Brereton, M., Watson, D. M., & Roe, P. (2019). Listening to Save mentioned. If you can think of ways of making this 

Wildlife: Lessons Learnt from Use of Acoustic Technology by a Species Recovery Team more clear please let me know 

Proceedings of the 2019 on Designing Interactive Systems Conference - DIS '19 

https://doi.org/10.1145/3322276.3322360 


m 
60 


ADDITIONAL ETHICAL CONSIDERATIONS Ethical considerations: | wonder, how we | have seen in project finders that projects are added by people not necessarily closely involved, | Thank you for this comment, I've given this a great Comment 
make sure that people running the initiatives become aware of it being added? and this leads to inaccurate information being in there, but if project leaders have no awareness | deal of thought too and hadn't included explicit incorporated 

they may not know it needs correcting. Alternatively, what if a project doesn’t want to be listed? | information in this version about that scenario. The into Beta 

| know this to be true for a few community-driven projects that don’t want to recruit broader challenge is to stop the more powerful from 

interest, but would rather keep the project very locally focused without being contacted but censoring the voices of the less powerful, but also to 

those outside of the community. prevent baseless slander etc (the challenge of all 
societies really!). | have now corrected this in the 
main text and further elaborated in the 
supplementary In future versions it will be necessary 
to further develop a transparent process if a report 
has been created about an initiative with no 
involvement from anyone associated with the 
project, or only one subset of stakeholders. In such 
cases, the Editorial team might give a standard 
period of time for any other stakeholders to be 
involved in checking and editing any information 
(similar to the concept of ‘right of reply’) before the 
report is published, or given the status ‘human 
reviewed’. However, the process for deciding which 
stakeholders to contact and how to ensure equity 
(alongside capacity considerations of the Editorial 
team) is an area for active discussion and 
development in future versions. For example, 
ensuring Indigenous peoples who are stakeholders in 
a mining initiative have been involved in checking a 
STARDIT report about a mining initiative created by a 
company with a declared financial interest in the 
mining initiative might present challenges that it is 
not possible for the Editorial team to overcome. Such 
challenges could be labelled and incorporated as 
future data categories in the STARDIT reports, for 


Feedback with an action for the paper 


| — 


18 | FOLKSONOMY VS ONOLOGY | also wonder if there has been any consideration to 
creating mechanisms for a folksonomy approach to tagging projects? | have 
suggested with the US and Euopean groups for citsci both. 


General feedback about STARDIT 


CITSCI GROUPS | WORK WITH DOING SIMILAR THINGS & ASSOCIATED RESOURCES WORK 
CONSIDERING CITING | have already directly shared this info with Jack and put him in contact 
with associated leaders, but just so you have it all in one place, citsci initiatives worth including: 
SciStarter [global citsci project finder]: https://scistarter.org/ Australian project finder: 
https://citizenscience.org.au/ala-project-finder/ [I helped facilitate SciStarter & ALA developing 
an API] The EU Citizen Science Cost Action CA15212 [https://cs-eu.net/] and it’s working group 
on Interoperability Working Group [https://cs-eu.net/wgs/wg5]; The working group’s outputs 
are reports [https://cs-eu.net/wgs/wg5/resources] and for context, | contributed to the 
workshop in Geneva and subsequent report “On the citizen-science ontology, standards & data 
[https://cs-eu.net/news/workshop-report-wg-5-geneva-declaration-citizen-science-data-and- 
metadata-standards]. The initiative recently rapped up and an associated book was published 
[https://cs-eu.net/news/book-science-citizen-science]. Several chapters may be of interest but 
around data and metadata efforts for citsci, see the chapter led by Rob Lemmens. | am currently 
working with him, Xeni, and Ina to progress some testing of the standards based on existing 
projects (included Jack in an email including them and Luigi as the COST Action working group). 
There is also a citizen science community of practice on data interoperability that is ending this 
month through the EU initiative, WeObserve. There is a lot of overlap in people with the COST 
Action group. See the resources created on their page: https://www.weobserve.eu/weobserve- 
cop3-interoperability-and-standards-for-citizen-observatories/ From the US there is the CSA 
Data and Metadata working group [be sure and see overhead tabs too: 
https://www.citizenscience. org/get-involved/working-groups/data-and-metadata-working- 


Change made 


example labels such as ‘report not checked by all 
stakeholders’ could be updated if more stakeholders 
have subsequently been involved in checking any 
report.’ 


from my understanding of the term ‘folksonomy’ 
(which | only learned from your comment!) both 
Wikidata and therefore STARDIT are a form of 
folksonomy. However, as | consider it a jargon term, 
I'll add the reference only and have incorporated the 
main points in into a minor re write of this section 
pasted below. Perhaps it is something that can be 
explored in more detail in future versions of the 
manual, as it's important to highlight this aspect of 
STARDIT. 'The current Beta Version of STARDIT maps 
terms and concepts using the Wikidata initiative 
(part of the Wikimedia Foundation)36, which 
includes definitions (taxonomy), a way of describing 
relationships between concepts (ontology)37, and a 
system to translate definitions and ontology between 
many languages. Examples of existing taxonomies 
include the National Library of Medicine’s Medical 
Subject Headings (MeSH), which are used extensively 
in multiple kinds of literature reviews 38. 

How to involve people in combining or merging 
overlapping taxonomies for different subsets of data 
has been identified as an important question in the 
process of taxonomy development 5960. By using 
Wikidata, STARDIT can be used by anyone store both 
public domain data and metadata (data about data), 
and link to hosted structured linked data. While it is a 
novel element set, where possible it will also 
incorporate element sets from established data 
standards and map them where possible (see Table X 
in appendix for examples of data standards which 
could be incorporated). This includes standard 
elements and value sets and controlled vocabularies 
61. The terms used in this paper are working terms, 
which will be progressively standardised over the 
lifetime of the project. 

Structured Wikidata can help define terms and 
concepts clearly and unambiguously, in a transparent 
and open way. 


Thank you - any new projects will be added to table 5 


Lead author's response if no 
change made 


Second Author Response 


Status 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Feedback with an action for the paper 


9 | Two other potential examples for health and social care include: process for 
identifying patterns of sub-optimal service, patterns for evaluating service 
improvement initiatives 


is} 


Building on Figures 2-3. Possibly a Flow chart visual to support 
readers/practitioners in the development of data for a STARDIT Report using the 
microcategories? For example, a chart breaking down steps and possible decision 
points with refs to sections in Table 4, for example. 


3 | (2) Due to the size of the STARDIT document — possibly divide into 3 separate 
standalone documents: 
About STARDIT, MANUAL, RESOURCES. 


3 | Table 5 resources to separate into thematic/domain sections based on field 
categorisations? Standardise these themes possibly aligning with Tables 2 and 4 in 
Manual. 


(4) Consider expanding participatory action research section with co-design and 
related methods of engagement? See some suggested resources below. 

- Line 38 - STARDIT (Standardised Data on Initiatives) ‘R’ needs to be bold?Also see: 
Lines 69, 105 


- Line 124 - The word stakeholders includes the public.. = use single quotes? 
=‘stakeholders’ 


- Line 142 - used in health, environment, manufacturing , publishing, government 
policy, education, arts and international development - link to Table 1. ? 


General feedback about STARDIT 


group/], and a subset of that group, including myself has been exploring development of the 
PPSR Core standards. | also CCed Greg Newman as chair of that group and Brandon Budnicki 
who is largely responsible for pulling all of our years discussions to date into creating the 
publicly accessible website very recently with ability to contribute via GitHub. PPSR Core: A Data 
Standard for Public Participation in Scientific Research (Citizen Science) 
[https://core.citizenscience.org/] 


overall really impressed with where this is at and the progress that has been made since | last 
read this work. 


Lead author's response if no 
change made 


Change made 


Thank you, added 


Thank you - all your comments addressed and 
responded to 


| think this is a fantastic idea. Sadly it is out of my 
personal capacity at this stage to create that although | 
think this would be good for future versions. | have 
flagged this with Thomas. 


Thank you - yes | think the idea would be (once 
published) the main body of the document is the 
STARDIT paper and then the supplementary resource 
is the manual, which will be as a PDF with references 
and additional information 

Yes, this is a good idea. While there are categories at 
the moment, perhaps a further level of 
categorization would be helpful? The amount of 
work to do this means it might need to be something 
for a future version but | will investigate this. 


Corrected thank you 
Thank you, done. 


Thank you, done. 


Second Author Response 


Agree it could be good. Alternative (or related) 
implementations could include: 

- In the technical manual a summary of questions you 
should ask yourself for each section 

- A stardit report for this current startit beta 
community feedback process 

- An annotated video of the above stardit form being 
filled out 


Status 


Comment 
incorporated 
into Beta 


comment 
noted for 
future versions 


Comment 
incorporated 
into Beta 


comment 
noted for 
future versions 


Comment 
incorporated 
into Beta 
Comment 
incorporated 
into Beta 
Comment 
incorporated 
into Beta 


Feedback with an action for the paper 


+ Line 288 - Across all disciplines, ‘plan’, ‘do’ and ‘evaluate’ are recognised as 
distinct stages of initiatives. Consider use of ‘PDSA’ — Plan, Do, study, Act ? a 
standardised iterative, four-stage problem-solving model. Check NHS publication: 
https://improvement.nhs.uk/documents/2142/plan-do-study-act.pdf 


- Line 326 - Table X summarises questions - Table X? 


N 
w 


ADDITIONAL RESOURCES ON FRAMEWORKS, etc 


N 
o 


‘Proposed policy (manifestoes)’ = should be: manifestos 


N 
o 


Suggestion to organize using a knowledge organization system? Consider Dewey 
Decimal 10 main classes to organize themes as outlined in table. 


N 
w 


‘cultural heritage ‘ : Suggestion to add cultural heritage to Arts section or on its 
own? - Large category 

e Tangible cultural heritage: movable cultural heritage (paintings, sculptures, coins, 
manuscripts) immovable cultural heritage (monuments, archaeological sites, and so 
on) underwater cultural heritage (shipwrecks, underwater ruins and cities) 

e Intangible cultural heritage: oral traditions, performing arts, rituals. 

Consider revised header: ‘Information, media and cultural heritage’ — to : 
Information, media and local traditional knowledge. See comment above 


Referring to Table 3 Questions for mapping preferences for involvement p. 17: 


To establish which group the person identifies as being part of — for example 
‘researcher’ or ‘participant’ 

Is there a table of definitions? There are several definitions in the introduction but 
these may be better highlighted in a table. Also note this Line 122 which suggests a 
blurring across some definitional boundaries? 

Referring to Table 4: Summary of STARDIT Beta Version data fields 

p. 21 Microcategory Section on ‘Methods’ - perhaps breakdown further e.g. 
quantitative, qualitative? Examples under each ? 

Link to methods of approach, such as PAR, co-design, referring to relevant 
sections... 

Why only ‘involvement (line 247)? In our systematic review/qualitative evidence 
synthesis (https://aagts. brasilia.fiocruz.br/wp- 
content/uploads/2020/10/Relatorio_POPART_final.pdf) on models and methods of 
social participation in R&D, policy, HTA, monitoring decision-making, we outlined 
engagement as the more comprehensive terminology, following, especially: 
Woolley, J.P., McGowan, M.L., Teare, H.J.A. et al. Citizen science or scientific 
citizenship? Disentangling the uses of public engagement rhetoric in national 
research initiatives. BMC Med Ethics 17, 33 (2016). 
https://doi.org/10.1186/s12910-016-0117-1 


N 
w 


BR 
R 


R 
R 


| understand the use of 'co-creation' here (line 253) and it is obviously adequate for 
the communal development approach deployed for STARDIT, but why not outline 
Sheila Jasanoff (2004) 'co-production' conceptualisation? It brings the systemic idea 
that there is the co-production of a culture that sustains and promotes the scientific 
knowledge productions and technological innovation while feedbacking the culture 
for STARDIT. 

line 263: | am happy to support this development and future development phases 
as well as operationalisation stages of STARDIT - especially regarding the proposed 
conceptualisation/terminology development abovementioned, if there is interest. 
Table 1 line 281 - Research reporting: Why is there not ‘data analysis' as a 
descriptor? This is a key ‘research’ activity to which | have been most often been 
asked about when | talk for experts and general public audiences to demystify 
misconceptions about citizen science around Brazil (especially high-ranking officers 
at the MoH and decision-makers at research foundations in Sao Paulo) [entire 


n 
R 


RB 
R 


General feedback about STARDIT 


Change made Lead author's response if no 


change made 


Thank you, very helpful illustration - I have added 
this reference to a systematic review calling for 
standardised reporting of PDSAs: 
https://qualitysafety.bmj.com/content/23/4/290.s 
hort (your link was expired) 

Corrected thank you 


Thank you- novel references will be added into text 
or Table 5 


Changed to 'Proposed policy (including draft policy 
and manifestoes)' 


Good suggestion, noted for future versions as no 


capacity to implement at this stage 


Thank you - rejigged categories and created tangible 
cultural heritage, using UNESCO terminology 


Thank you - reworded 'To establish which grouping(s) 
the person identifies as being part of — for example 
‘researcher’ or ‘participant’ (noting any groupings 
should be co-defined)' 


Change category to ‘Methods and paradigms' - 
recognising that participatory action research and 
associated terms might be considered guiding 
paradigms not strictly methods themselves. Also 
added quant and qual as an e.g. 

Thank you for this comment, the confusion around 
terms such as ‘involvement' and 'engagement' is 
central to this paper and while it acknowledges there 
are different terms for the same thing, this paper 
articulates what this paper means by involvement in 
the section 'While meanings of these terms are often 
imprecise and can be used interchangeably, 
‘involvement’ here is distinct from ‘engagement’, 
which is where which information and knowledge 
about initiatives is shared, for example, with study 
participants who remain passive recipients of 
interventions’ - however | have added the ref 
'Disentangling the uses of public engagement 
rhetoric in national research initiatives’ 

Ref added to supplementary information as we can't 
say co creation was informed by this but I've said 
‘Future co-creation processes should be decided 
collaboratively’. 


Thank you - support gladly recieved! 


Thank you, added 


Second Author Response 


Status 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 
Comment 
incorporated 
into Beta 
Comment 
incorporated 
into Beta 
comment 
noted for 
future versions 
Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 
Comment 
incorporated 
into Beta 


| | Respondee 10 


14 


R 


R 


R 
R 


nN 
a 


N 
a 


Feedback with an action for the paper 


research process]: ‘data validation’ is also a concern for professionaly-trained 
researchers and decision-makers around Brazil. - as per my previous comment. 
Coding and algorythms reporting: Information about Data Privacy, Laws, 
Regulations, Directives and data security (not only pertaining to data ownership) 
should be mentioned here as several countries either follow GDPR, HIPPAA, GINA. 
and Brazil, for instance adapted GDPR to its own Lei Geral de Proteção de Dados 
that came into effect Sept/2020. 

Other services: Jasanoff (2004) co-production conceptualisation here would be very 
beneficial as it outlines in which ways STARDIT can contribute to both identification 
and geolocalisation of impact indicators that should inform in real-time (ongoing 
manner) about indicators that are context-specific and may be used to develop 
glocally-relevent, local, regional, national indicators on various issues that remain 
as an unmet need somewhere, which requires data-intensive publications - it's the 
concept of change the culture behind the research-to-practice gap. 

involvement' (line 307): Again, do reffer back to our qualitative evidence synthesis 
as you may find how we have updated Rowe & Frewer's (2004) typology from 
Arnstein's ladder of social participation to outline that engagement is a more 
comprehensive terminology when it comes to citizen science and active research 
methods: https://aagts. brasilia.fiocruz.br/wp- 
content/uploads/2020/10/Relatorio_POPART_final.pdf line 

337: OTHER OPTION: people who are not affected directly or indirectly but who 
hold citizenship to a certain region, country, municipality, neighbourhood and has 
interest in participating in decision/policy-making and/or conttributing with R&D 
that might be of relevance to improoving the livelihood of fellow citizens from the 
same region, country, municipality, neighbourhood. 

line 169 - REFERENCES: | have looked through your references and | see that you 
are familiar with Muki Haklay's amazing work on ‘extreme citizen science’, but | 
found that you may not be aware of Barbara Prainsack's work on citizen science 
(she had proposed in this chapter from 2013 that | have revised something very 
similar to what you propose here with STARDIT, but not as developed as you have 
done - Prainsack, B. Understanding Participation: The ‘citizen science’ of genetics. 
In: Prainsack, B., Werner-Felmayer, G., Schicktanz, S. (eds). Genetics as Social 
Practice. Farnham: Ashgate. Available at: 
https://www.researchgate.net/publication/236850804_Prainsack_B_Understandin 
g_Participation_The_'citizen_science'_of_genetics_In_Prainsack_B_Werner- 
Felmayer_G_Schicktanz_S_eds_Genetics_as_Social_Practice_Farnham_Ashgate) 
and other publications - you may want to ask her to review STARDIT (I can mediate 
connection, if need be). 

1) The definition of “initiative” is a bit novel — are there ways of highlighting what is 
meant by it in the context of STARDIT? One suggestion would be to have it 
explained in a text box, where additional level of detail and examples could be 
given for those that need it without disrupting the flow of the text. 


1) I think STARDIT has a lot of potentially very significant contributions that could 
really make a difference, but | find it a bit hard to pick them out from the text. 
Would it be possible to make a box just highlighting the (potential) benefits/value 
added of STARDIT? This would be in addition to the table on the applications of 
STARDIT in different disciplines, and basically just be X number of bullet points 
explaining what STARDIT can do and why it is important. 

3) It might be helpful to go into more detail, if possible, of the benefits of getting 
standardized data about initiatives? |.e. what does it help us to know the values of 
people involved in a policy initiative. 


4) The abstract seemed to me to be very strongly focused on methodology (citizen 
science and participation action research), but as | understood it STARDIT goes well 
beyond this. The phrasing in para two in the abstract (line 64 onwards) seems to 
me to suggest a delimitation of STARDIT to standardised data about who and how 
people are involved in initiatives, and | feel that underscores the potential 
contributions of STARDIT. 


5) A lot of initiatives will focus on evaluating another initiative, and it would be 
great to cross-reference all initiatives related to a specific topic or theme, especially 
to see how outcomes are assessed by different initiatives. | might have missed this 
in the paper, or the added material, but | think a way to examine the results of all 
evaluations of outcomes for project X would be a great benefit, including for policy 


General feedback about STARDIT 


Change made 


Thank you - added to 'Management and monitoring’ 
section 


added ', process for identifying impact indicators 
(including geolocation data)' 


Thank you -as per previous response this is covered 
in detail and we're using consistent terminology in 
this article. Have added ref - please check it's correct 


Thank you for offer of connection with Barbara - only 
just seen this - yes please for future versions! 
Reference added to table 5 


added ‘Transparent acknowledgement of differing 
values and perspectives is critically important, in 
particular when mapping if different stakeholders’ 
values are complementary or opposing’ 


thank you - reworded for clarity 


added sentence ‘In addition it allows comparison of 
both evaluation methods and any impacts or 
outcomes in relation to standardised terminology.' 


Lead author's response if no 
change made 


this distinction. These categories are fairly fixed in this 
now as they were used accross the preference mapping 
for the alpha version but noted for future versions 


| feel this is hopefully this is covered in the section 
‘Defining ‘initiative’ and ‘involvement’ - but a list of 
terms used in the paper could be a useful additional 
table. For consideration 


Noted - for consideration 


Second Author Response 


Status 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


comment 
noted for 
future versions 


comment 
noted but not 
incorporated 
into this 
version 
comment 
noted but not 
incorporated 
into this 
version 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Respondee ID 
N 
a 


Feedback with an action for the paper 


and future option parts of regional and global assessments, such as those produced 
by IPBES and the IPCC. 


Accurate and reliable data is of critical importance for all planned, collective human 
initiatives (see box 1). Whether we are implementing government policies, 
undertaking research, or developing new industry, better decisions improve 
outcomes — and the quality of our decisions is inextricably linked to the quality of 
the information we have at our disposal. 


The amount and quality of data we need to make good decisions is increasing 
exponentially as the problems we are trying to solve become increasingly complex, 
global and and cross-sectorial. This is reflected in the range of on-going efforts to 
mobilise, standardise and share data in a number of fields. However, what is 
currently lacking is a standardized way to share information about initiatives 


themselves. What was the objective, who were involved, who did which tasks, what 


methods were used, what was the governance structure, which results were 
achieved? Answers to all of these questions contain important data that could, and 
should, provide valuable insight and inform design and implementation of future 
initiatives. STARDIT’ (Standardised Data on Initiatives) aims to address this gap by 
standardising and sharing data about human collective actions. 


STARDIT is a free open access data sharing system that works across fields, 
disciplines and languages. Data about initiatives can be submitted by anyone, and 
updated throughout the lifetime of an initiative, from planning to evaluation and 
reporting any impacts. The authors of the data can be verified, and the data 
assessed for quality. STARDIT is being developed on the understanding that the 
complex global problems we are facing today require evidence-informed 
collaborative methods, multidisciplinary research and interventions in which 
affected population groups are integrally involved in every stage. 


Among its main benefits (see box 2 for further information), STARDIT offers those 
carrying out research and interventions access to standardised information which 
enables well-founded comparisons of the effectiveness of different methods. 
Uniquely, STARDIT also enables sharing of information about stakeholder 
involvement in initiatives that works in many languages, using the Wikidata system. 


This article outlines how STARDIT works and how contributors from multiple 
disciplines and organisations globally might continue to be involved in the 
development of the current Beta Version. 

44-47: Lines 38-39 gives a very wide definition of what is understood by ‘initiatives’, 
but lines 44-47 can be read as only linking to one set of problems (i.e. complex 
global problems). Would it be possible to open this up slightly to stress from the 
very beginning the wide possible application and relevance of STARDIT? For 
instance: 


The authors of the data can be 

45 verified, and the data assessed for quality, offering a potentially important 
source of high-quality 

46 standardised information on initiatives trying to improve complex global 
problems (responses to which 

47 transcend the capacity of any single discipline). 

59-63: Is distinctions the right word here? Maybe rewrite to something like 
“sectorial and disciplinary barriers can limit”? 


64-65: | would perhaps not lead with this focus on citizen science and participatory 
action research, but if we do it would probably be best to rewrite this part: “crucial 
methods to use to solve” 


General feedback about STARDIT 


Change made 


Thank you for this fantastic plain english summary. | 
have incorporated some words. Noting the word 
limit | will save this plain english summary to try and 
incorporate more if possible. 


Thank you, changed plain english summary: ‘There is 
currently no standardised way to share information 
across disciplines about initiatives, including fields 
such as health, environment, basic science, 
manufacturing, media and international 
development. All problems, including complex global 
problems such as air pollution and pandemics require 
reliable data sharing between disciplines in order to 
respond effectively. 


changed to 'As we face increasingly complex 
problems, such as global air and water pollution, 
disciplinary and sectorial distinctions can limit our 
ability to respond effectively’ 

rewritten as 'Current reporting methods lack 
information about the ways in which different people 
are involved in initiatives, making it difficult to collate 
and appraise data about the most effective ways to 
involve different people. For example, forms of 
participatory action research where anyone can be 
involved in any aspect of research (including ‘citizen 
science’) are increasingly recognised as crucial 
paradigms for solving global problems, as they can 
help ensure that initiatives are aligned with the 
priorities of those affected, thus blurring the lines 
between concepts such as ‘researcher’, ‘public’, 
‘patient’ and ‘citizen’. ' 


Lead author's response if no 
change made 


Second Author Response 


Status 


comment 
noted for 
future versions 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Feedback with an action for the paper 


N 
a 


66-68: Does it also miss other things that STARDIT provides? Highlighting that here 
might help make the value added of STARDIT even clearer. 


69-71: “Standardised data can inform effective ways to share power during the 
design...” sounds a bit strange to me —| understand what is meant, but it probably 
isn’t quite correct (e.g. might need to specify which standardised data) and could 
probably be rewritten to make it more precise. 

75-76: Is it possible to be more precise? Or, alternatively, sound more precise than 
“many kinds of data”? 


nN 
a 


86-88: Would it be better of simply having an annex with a list of everyone that 
participated, and their affiliation, rather than highlighting a few organizations? It 
comes across as a bit strange to me (possibly because | do not understand these 
organizations are highlighted — but that will probably be the case for most readers). 
95-98: This is so well established that it might be better to not mention any specific 
organization, but just to have a several of the most relevant references for it. Then 
you could also remove the “For example”. 


N 
a 


100-102: The sentence starting with “A scientific...” seems to be a put on its head 
somehow, especially with the part saying “in many contexts”. Maybe write: A 
scientific evidence-informed approach is often the most appropriate model for 
analysing the effectiveness of interventions. 

100-104: This para is a bit unclear to me — both the way it is phrased and its 
message. As | read it the message seems to be: 1. People need access to valid and 
reliable information. 2. Analysing the effectiveness of interventions typically 
requires an evidence-based approach. 3. This evidence-based approach goes under 
many names, including evaluation, international development, education or 
initiative. 4. We use the term initiative to refer to all of the above. 

109: The united Nations secretary-general stated that “..... 


N 
a 


nN 
a 


Y 
a 


115: Is this wording a bit strong? Because the ‘initiative’-category is so large it 
probably contains a lot of examples of initiatives where inclusion it is not strictly 
“essential”? 


N 
a 


117: What does “international consensus statement” mean? Could it be spelled out 
or explained in the sentence? 


N 
a 


123-124: Would it be possible to clarify what is meant by “STARDIT can report any 
different ‘interests’ and ways of sharing power among different stakeholders”? 


nN 
a 


129-131: Substitute meanwhile with another word? 


132: Convoluted sentence? Change to: Sharing data in a consistent manner may 
help ensure that benefits of initiatives are shared more equitably? 

141: Move “is”, so that the sentence says: ... a culture of partnership across 
disciplines and is, whenever possible, aligned and...” 


nN 
a 


142: Specify that these are examples? 


General feedback about STARDIT 


Change made 


rewritten as ‘Standardised data can inform effective 
ways to share power during the design, 
implementation and evaluation stage of initiatives. 
For example, when designing a response to an 
epidemic, standardised data can improve retrieval of 
relevant information which can be used to inform 
which affected individuals or organisations could be 
involved in the design of the response and which 
outcomes are most important.’ 

reworded to ‘Standardised data can inform effective 
ways to plan, implement and evaluate all stages of 
initiatives. ' 


reworded to 'STARDIT will enable multiple categories 
of data to be reported in a standardised way across 
disciplines, facilitating appraisal of initiatives and 
synthesising evidence for the most effective for 
people to be involved in initiatives. ' 

good point. | added them for credibility but 
highlighting only a few is potentially sending the 
wrong message. removed. 


agreed. also reworded to ‘Many problems facing life 
on earth transcend the capacity of any single 
discipline to address’ so it's not so human specific in 
relation to the problems 


Excellent pick up, done. 


thank you - | have reworded as suggested. 


thank you, corrected 


changed to 'It is often essential’ noting probably 
better wording is ‘always best practice’ - but that's 
wordy 

now just called it a ‘statement’ 


I thought about this a lot and added this sentence 
with a ref: An interest can include a kind of 
commitment, goal, obligation, duty or sense of 
connection which relates to a particular social role, 
practice, profession, experience or medical 
diagnosis 31. | also tried to create a wikidata entry 
as | don't feel there is one currently: 
https://www.wikidata.org/wiki/Lexeme:L483913 
changed to ‘Other examples include’ 


changed as suggested 
changed as suggested 


changed to ‘such as those used in health, 
environment, manufacturing , publishing, 
government policy, education, arts and international 
development ' 


Lead author's response if no 
change made 


Second Author Response 


Status 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 
Comment 
incorporated 
into Beta 
Comment 
incorporated 
into Beta 
Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 
Comment 
incorporated 
into Beta 
Comment 
incorporated 
into Beta 
Comment 
incorporated 
into Beta 


Feedback with an action for the paper 


N 
a 


143: Be specific about how it works across human languages or is this evident to 
all/most? 


147: Sounds slightly underwhelming — could the benefits be highlighted better? 


nN 
a 


222-223: What exactly would this entail? Sounds very ambitious. 


N 
a 


234-235: How are they mapped and reported? 


nN 
a 


415: Change to: is expected to take at least 5 years, and will likely involve? 


438-439: are is paramount 


440: in the future 


1. The current text uses terms such as “intellectual property”, “ownership”, 
“authorship”, and “license” in a way that is ambiguous, misleading, and internally 
contradictory. Unfortunately, if this critical issue is not clarified early on (i.e. now), 
it will create a ticking time-bomb that will go off at a later point during this project. 
| have been involved in open science initiatives for many years, co-founded a citizen 
science project, edited a guide on best practices for researchers, and received 
official certification from the Creative Commons organisation on copyright and 
licensing. In addition to my specific comments further below, please let me know if 
and how | can assist in fixing this problem to enable the wider sharing and 
implementation of STARDIT that it clearly deserves. 


N 
R] 


N 
A] 


2. | respectfully take issue with the claim that STARDIT is useful for “any type of 
initiative, across any discipline” (line 278). Reading the text and tables (such as 
Tables 1 and 5), it seems that STARDIT is heavily informed by public health, citizen 
science, education, environmental, and/or international development initiatives. 
While commendable, they do not encompass “any discipline”. For example, would 
STARDIT be useful for an astrophysicist, chemical engineer, archaeologist, or 
historian? If so, how? If not, then the wording of “any initiative” or “any discipline” 
should be changed and constrained. 


N 
A] 


3. Broadly speaking, | would like to see more details on how STARDIT will continue 
to evolve, adapt, and improve after version 1.0. 


General feedback about STARDIT 


Change made 


sentence reordered to ‘The working Beta Version of 
STARDIT uses Wikidata to enable definitions to be co- 
created by contributors anywhere in the world, and 
therefore works across human languages, with 
interoperability with other platforms planned for 
future versions’ 


whole section rewritten 


added example ' For example, STARDIT has already 
been used to map the varying perspectives of 
multiple stakeholders when planning a multi- 
generational cohort study 73' 

reworded to ‘The participatory process used for 
developing STARDIT has attempted to be transparent 
about how different stakeholders have been involved 
in shaping it in order to improve how the system can 
be used to map values and provide more culturally 
neutral guidance for planning and evaluating 
involvement in initiatives. ' 

changed to ‘amassing sufficient reports to create a 
useful database is estimated to take at least 5 years, 
and will likely require machine learning’ 


Any further support in correcting this would be much 
appreciated 


more information has been added to the 
supplementary section, noting that providing any 
specifics past version 1 would be inconsistent with 
the co-creation process - but the values which guide 
it would point towards many more versions we 
would hope. 


Lead author's response if no 
change made 


| would say that yes, STARDIT could be very useful for all 
those other diciplines and initiatives, although naturally 
the development so far has been by people from health 
and enviroment, we want to ‘leave the door open’ to all 
diciplines. While we have provided mulitple examlpes in 
the table of how it could be applied, | acknowledge this 
table is not exhaustive. | have however attempted to 
include data management (part of astrophysics, chemical 
engineering) and also cultural management in there too. 
I've yet to think of a collective human action that 
couldn't be explained with a STARDIT report, so! 
respectfully acknowlege this point and will keep the text 
as is. 


Second Author Response 


Status 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 
Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 
Comment 
incorporated 
into Beta 
Comment 
incorporated 
into Beta 


comment 
noted but not 
incorporated 
into this 
version 


Comment 
incorporated 
into Beta 


N 
A] 


N 
R] 


N 
A] 


Feedback with an action for the paper 


4. Exactly who is the intended audience of this paper? As described in the “Beta 
Version interface” section, substantial technical skills and knowledge (such as 
familiarity with the usage of APIs, data science, RDF structure, etc.) are needed to 
make full use of STARDIT. Therefore, | don’t think literally anyone running an 
initiative can just jump in. Are there plans to make STARDIT more accessible and 
useable? If so, to whom and how? 


5. There is terminology not accessible to a broad, non-technical audience. For 
example, Wikidata is not defined on its first use on line 144. 


It is not clear what the “Preferred Reporting Items for Systematic Reviews and 
Meta-Analyses” is which should at least come with a citation. 


Importantly, the Internet Archive is mentioned several times without explaining 
what it really is and why it is used. 


Also, ORCID is not clearly defined and will be confusing for a non-technical reader. 
These things should be clarified. 


The document emphasised a desire to make STARDIT “always be open access”. | 
wholeheartedly support the general sentiment, but the terminology and concepts 
employed throughout the text are unintentionally but highly misleading and 
sometimes contradictory: 

The term “public domain” is used many times throughout the text. “Public domain” 
has a specific legal definition meaning something that is without copyright. 
According to national laws and international agreements such as the Berne 
Convention with 179 signatories, copyright is automatically applied to any 
intellectual work at the moment of creation with no way to opt-out. Copyright gives 
the copyright holder monopoly privileges and powers to dictate how the 
copyrighted work is to be used. This automatically applies to STARDIT and any other 
information/material mentioned in the current document. Colloquially, “public 
domain” might not be used with such specific meaning, but in the context of this 
STARDIT Beta version manuscript — which explicitly mentions “open access” and 
licensing — it is imperative to use unambiguous language. There are too many uses 
of “public domain” throughout the current text, but | strongly suggest replacing 
those instances with “publicly accessible”, “in public view”, or “in the public” as 
appropriate (unless, of course, when the intention is to refer to the legally-defined 
term “public domain”). If additions have been made after the version at 
https://osf.io/w5xj6/, | suggest running a search-and-replace operation to find and 
remove ambiguous uses of “public domain”. 

Line 91 claims that the STARDIT beta version is released under “a Creative 
Commons license.” There are six separate Creative Commons licenses with vastly 
different implications on how the licensed work can be shared and used. Please be 
clear about which one STARDIT uses. Is it the Creative Commons Attribution license, 
the Creative Commons Attribution-ShareAlike license, or another one? This point 
applies to other mentions of these licenses throughout the rest of the text. On that 
note, a license is granted by the copyright holder to others who wish to make use of 
a work. If something is in the public domain, then by definition no license can be 
applied to it. 

In addition, please avoid the ambiguous term “intellectual property” which has no 
specific definition. Generally speaking, “intellectual property” includes legal 
concepts such as copyright, patents, trademarks, trade secrets, among others. Text 
such as this manuscript or software code are mostly covered by copyright while 
physical designs like that for a machine are commonly dealt with through patents. 
Logos, such as the one proposed for STARDIT, are often handled through trademark 


General feedback about STARDIT 


Change made 


definition of wikidata included in objectives 


citation added and comma added to clarify it's a 
definition of what is 'Future versions should be 
informed by a regular, systematic search, review and 
appraisal processes, using the Preferred Reporting 
Items for Systematic Reviews and Meta-Analyses 
(PRISMA) data set, used for reporting in systematic 
reviews and meta-analyses." 

definition added and ref 


This is incredibly helpful feedback. I've replaced all 
uses of ‘public domain' with ‘publicly accessible’ and 
note that these terms have been used incorrectly in a 
colloquial way (as I'm not qualified in this area) and 
this terminology absolutely needs to be 
unambiguous. 


Thank you for this detailed and valuable feedback. | 
think the entire STARDIT project should take further 
advice on this, but for now the most sensible 
decision | feel is going with a Creative Commons 
license and reviewing any next steps for version one 
- as this step gives us some control over usage of 
logo and other work associated with this project, 
without it being in danger of becoming a 
commercial commodity that could be ‘bought out' 
as it were. My instinct is to go with this one 
(Attribution-NonCommercial-NoDerivs CC BY-NC-ND 
- https://creativecommons.org/licenses/by-nc- 
nd/3.0/au/) currently as | think not allowing anyone 
to change it would hopefully prevent the project 
‘forking’ - centralising the control of what STARDIT 


Lead author's response if no 
change made 


"In short, yes | agree some of the language is very 
technical two parts acknowledge we need to be more 
inclusive (all, pending funding!) - ' and may require 
creating additional tools to create more inclusive ways of 
involving people in developing taxonomies.40' and ', 
ongoing co-design will be required to ensure STARDIT is 
as accessible and inclusive as possible. ' - which | hope 
are sufficient 


It's hard to answer who is the audience. In summary 
probably anyone with a professional role in planning or 
managing an initiative, in particular those with a focus on 
involving people/citizen science " 


ORCID explained in full in table 4 and not used in text 
anywhere else in main body of paper 


Second Author Response 


Status 


comment 
noted for 
future versions 


comment 
noted for 
future versions 
comment 
noted for 
future versions 


comment 
noted for 
future versions 
comment 
noted but not 
incorporated 
into this 
version 
Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Feedback with an action for the paper General feedback about STARDIT Change made 


registrations. Please be specific. 

To be clear, open access usually does not mean a lack of copyright and being in the 
public domain. Open access is very much enabled by using open licenses (such as 
among the six Creative Commons licenses) to expressly grant freedoms to share 
and reuse information while requiring attribution. On a more pedantic note, it is 
possible to use the CCO Public Domain Dedication to explicitly relinquish all 
copyright associated with a work and it would still technically be considered open 
access. However, this also means that no attribution is necessary when the work is 
being shared and reused, so that might not be desirable in many cases. 

| recognise that lines 230-233 states the desire to avoid imposing a set of values on 
what constitutes “ownership”. However, STARDIT explicitly adopts the Creative 
Commons licenses which operate within a copyright regime that has 
(unfortunately) been imposed on almost the whole world. Therefore, | stress again 
that terminology around copyright and licensing be made clear and unambiguous. 


is inside the steering committee (and any other 
subsequent governance processes decided upon) 
rather than having a free-for-all, which could work 
against the very thing we're trying to achieve 
(standardisation!). Multiple derivations of STARDIT 
would be a nightmare! 


Line 91 — Replace “a Creative Commons license” with “the Creative Commons x 
license” where x is the specific license that has been applied to STARDIT. 


I have added information including this sentence 
'STARDIT and all associated work and logos are 
currently licensed under Creative Commons license 
(Attribution-NonCommercial-NoDerivs CC BY-NC- 
ND), with the quality of any future iterations being 
the responsibility of not-for-profit host organisations 
and future licensing decisions to be made 
transparent, with anyone invited to be involved." - 
note | did not add specific information to abstract for 
word count reasons 

changed to this, noting that copyright etc is not a 
universally accepted construct or way of working 
‘(including concepts of intellectual property, 
copyright and license information, relevant 
blockchains and non-fungible tokens), evaluating 
knowledge translation, reporting impacts and 
outcomes 


agree this is a distinct category, added 


Table 1 sub-area “Coding and algorithms” — Replace “concepts of intellectual 
property and copyright” with “license information”. 


N 
A] 


Table 1 area “Information, media and cultural heritage” — Many initiatives also 
produce physical artefacts such as medical devices to treat malaria or a do-it- 
yourself solar cooker. These designs are also valuable information to be published. | 
suggest a sub-area titled “Hardware designs” with this “Relevant data categories” 
text: Reporting: Who created the designs, who reviewed them, what formats are 
the designs shared as and in what medium, information on license(s), outcomes 
and impact of the hardware. 


N 
A] 


Line 360 — Replace “public domain reports” with “publicly-viewable reports 


N 
R] 


e Line 366 — Replace “public domain URLs” with “public URLs”. changed to 'publicly accessible URLs ' 


N 
uv 


e Line 387 — Replace “will also be archived in the public domain” with “will also be 
archived in a publicly-accessible location online”. 


N 
A] 


e Line 393 — Replace “public domain sources” with “public sources”. 


N 
R] 


e Table 4 section “Initiative context” data category “Identifying information” — 
Replace “public domain URL with” with “URL”. 


N 
R] 


e Table 4 section “Initiative context” data category “Methods” — Replace “include a 
link to a public domain document” with “include a link to a published document”. 


e Table 4 section “Report authorship” data category “Identifying information for 
each author” — Replace “public domain profiles” with “public profiles”. 


Lead author's response if no 
change made 


Second Author Response 


Status 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 
Comment 
incorporated 
into Beta 
Comment 
incorporated 
into Beta 
Comment 
incorporated 
into Beta 
Comment 
incorporated 
into Beta 
Comment 
incorporated 
into Beta 
Comment 
incorporated 
into Beta 


N 
A] 


N 
R] 


N 
R 


Feedback with an action for the paper 


e Table 4 section “Data” - Rename this section to “Data, software code, and 
hardware designs”. 


e Table 4 section “Data” data category “Ownership and control” — Replace “Who 
‘owns’ the data or claims any kind of ‘intellectual property’ or rights (include 
relevant licensing information)” with “detailed licensing information”. 


e Table 4 section “Data” data category “Ownership and control” — Replace “public 
domain URL” with “public URL”. 


e Table 4 section “Information completed by Editoris” (misspelling of “Editors”?) - 
Replace “public domain URL” with “public URL”. 


e Line 485 — Replace “shared in the public domain” with “publicly shared”. 


e Page 28 section “Who is involved in STARDIT?” - Replace “all decisions made 
transparently and in the public domain” with “all decisions made transparently and 
in the public”. 


e Page 31 “Additional values and paradigms” — This section mentions “STARDIT 
design and code should always be open access and relevant licenses should always 
be Creative Commons”. One of the six Creative Commons licenses must be 
specified here. The Creative Commons Attribution 4.0 or Creative Commons 
Attribution-ShareAlike 4.0 licenses are the most often used in open science. In 
addition, there is a separate set of licenses used for software code such as the GNU 
General Public License (GPL) 3.0 and that should be specified as well. It is legally 
highly problematic to apply any of the Creative Commons licenses to code and it 
should be avoided. 


e Page 31 “Immutable values” — Replace “They will always be shared in the public 
domain” with “They will always be shared publicly”. 


General feedback about STARDIT 


Change made 


changed to ‘Data (including code, hardware designs 
or other relevant information)' 


I've kept it plain english but added brackets 'Who 
‘owns’ the data or claims any kind of ‘intellectual 
property’ or rights (include relevant licensing 
information)’ 


done and corrected, thank you 


changed to 'In plain English, anyone can get involved 
and have a say in how it should be designed and run, 
with all decisions made transparently and stored in a 
publicly accessible way.' 

Changed to '* STARDIT designs and code should 
always be open access and relevant licenses should 
always be those which allow others to build on and 
improve the project, while maintain central control 
over quality (such as the Creative Commons 
Attribution-NonCommercial-NoDerivs CC BY-NC-ND 
license and the GNU General Public License (GPL) 3.0 
for code)' 


changed to ‘While these values will evolve, we will 
keep an immutable record of our values. They will 
always be shared via a publicly accessible URL and 
regularly archived on the ‘Internet Archive’ for future 
reference 120.' 


Lead author's response if no 
change made 


Second Author Response 


Status 


Comment 
incorporated 
into Beta 
Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 
Comment 
incorporated 
into Beta 
Comment 
incorporated 
into Beta 
Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Respondee ID 
N 
R 


Feedback with an action for the paper 


e Page 44 line 17-20 (under “Data ownership and hosting” section) — Here it states 
a STARDIT report cannot contain any information that is not already in the public 
domain. If the current authors are referring to the legal term “public domain”, then 
this will be almost impossible. For example, the vast majority of open access 
scientific peer-reviewed publications are under copyright and released under one 
of the Creative Commons licenses (usually the Creative Commons Attribution 
license). By definition they are not in the public domain and the information they 
contain cannot be included in a STARDIT report. In addition, “to avoid ‘intellectual 
property’ issues” is highly ambiguous and misleading. If the current authors are not 
referring to “public domain” in the legal sense of the term, the | suggest replacing 
the first sentence of this section with “To reduce data sharing barriers and 
encourage reuse, a STARDIT report cannot contain any proprietary information that 
is not open access”. 


e Page 47 lines 110-115 — The current paragraph implies that Science for All will 
hold the trademarks for the STARDIT logos and the copyright over the STARDIT 
specification, this manuscript, and other outputs from the participatory action 
research process. Is this the case (e.g. did Science for All register the trademark for 
the logo)? If so, please be specific. Also, if the intent is for Science for All to hold the 
copyrights stated above, then that implies the authors do not. Is this also 
intentional? The copyright holder(s) and author(s) are not always the same people. 
I am happy to suggest edits to this important paragraph but cannot do so without 
more specific knowledge of what the intent is. 


e Table 5 — The “Access” column in this table makes a distinction between “public 
domain” and “open access”. This is confusing because material in the public domain 
counts as open access. 


General feedback about STARDIT 


Change made 


Thank you - does this work? 'To reduce data sharing 
barriers and encourage reuse, a STARDIT report 
cannot currently contain any proprietary information 
that is not open access or publicly accessible, except 
for information volunteered by the report authors 
(such as institutional email addresses), much like a 
“corresponding author’ on a peer reviewed paper. ' 


Thank you - working backwards, what I'm trying to 
achieve is a stepping stone stage - the end goal being 
STARDIT set up formally as its own thing - but for 
now it needs a host organisation, which is Science for 
All (as I'm Director, so it makes things simple). In the 
future, if it's another charitable/not for profit 
organisation which takes this over, great (e.g. 
‘Standardised Data International - STARDIT'). What 
I'm trying to prevent is a situation where someone 
uses the STARDIT name or logo and we have no legal 
way of preventing it (e.g used for commercial 
purposes or an extreme political group). So | think for 
now, for all practical purposes, Science for All will be 
the copyright holder until a better solution is found. 
I'm really no expert on this so legal advice would be 
very welcome 


Changed to 'For the purposes of concepts of 
intellectual property (including trademarks), and to 
protect STARDIT from being used by people in ways 
which are outside of the values defined in this 
document, any intellectual property (including logos 
or code associated with STARDIT) are currently 
owned by the charity Science for All, which is 
currently hosting the participatory action research 
process to create and manage STARDIT. Any 
decisions relating to the above (including transfer of 
ownership of any intellectual property) are to be 
made by the STARDIT Steering Committee, which 
hosted by Science for All, but independent of Science 
for All. In the future ownership may be transferred to 
an appropriate organisation established specifically 
for the purpose of owning such intellectual property, 
for example, establishing a charity called 
‘Standardised Data International — STARDIT).' 
corrected as either Publicly accessible website 

or Open access 


Lead author's response if no 
change made 


Second Author Response 


Status 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Respondee ID 


N 
A] 


Feedback with an action for the paper 


e Citizen science is briefly mentioned in the abstract but not defined or elaborated 
on in the text. | suggest just removing the mention of it in the abstract. Otherwise it 
needs to be defined in the text with relevant citations such as, but not limited to: 
Auerbach, J., Barthelmess, E. L., Cavalier, D., Cooper, C. B., Fenyk, H., Haklay, M., 
Hulbert, J. M., Kyba, C. C. M., Larson, L. R., Lewandowski, E., & Shanley, L. (2019). 
The problem with delineating narrow criteria for citizen science. Proceedings of the 
National Academy of Sciences, 116(31), 15336-15337. 
https://doi.org/10.1073/pnas.1909278116 

Silvertown, J. (2009). A new dawn for citizen science. Trends in Ecology & Evolution, 
24(9), 467-471. https://doi.org/10.1016/j.tree.2009.03.017 


e Line 415-416 (page 25 in Discussion section) — Why and how might machine 
learning be applied to STARDIT data? Without elaborating more this feels like a 
throwaway sentence. 


e Lines 97-100 on page 46 section “How does STARDIT work within ‘law’ and ‘lore?” 
(missing ‘ after lore?) - In addition to indigenous peoples, there are entire sovereign 
states that are not recognised by the United Nations. So, if “STARDIT conduct will 
be guided by the United Nations on all matters of law”, how will citizens of non-UN- 
recognised states be heard, represented, and included? 


General feedback about STARDIT 


Change made 


the term ‘citizen science’ is used in inverted commas 
in the abstract, and there's no space for explanation - 
I think the term is familiar enough to be used in this 
context in the abstract, although a more full 
explanation is now included in the background 
section with added refs 


added extra sentence and ref 'STARDIT seeks to be 
an easy-to-use way for people from multiple 
disciplines to share data about atives. However, 
amassing sufficient reports to create a useful 
database is estimated to take at least 5 years, and 
will likely require machine learning. For example, 
adversarial machine learning may be used in parallel 
with humans (for verifying data) to generate STARDIT 
reports from existing publicly accessible data at a 
scale and speed impossible for humans alone to 
achieve. ' 

Thank you - added this 'STARDIT will always defer to 
United Nations declarations in favour of any 
sovereign laws, and does not recognise itself as a 
legal entity which is acting in any one sovereign 
state. Citizens of non-UN-recognised states will be 
recognised as individual legal persons and included 
equally alongside any other person, regardless of 
status (including citizens, residents, asylum seekers 
and refugees). STARDIT is an initiative to support 
individuals to self-organise ways of sharing 
information, and it is the responsibility of each 
individual (including individuals working on behalf of 
organisations) to act both within the values of 
STARDIT, and any laws to which they may be subject. 


Lead author's response if no 
change made 


Second Author Response 


Status 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Comment 
incorporated 
into Beta 


Change log from Alpha Version 


Í 


Plain English 


summary 
Made it clearer in the phases section that initial audiences might need targeting 


Added use case for NGOs in table 1 examples of use and added this ref: 
https://www.brookings.edu/wp-content/uploads/2016/06/11_development_aid_kharas.pdf 


have added 'biodiversity loss' at later point, as habitat loss is a jargon term (some people might 
think it means places for humans to live) but this point is about how health and enviroment are 
interlinked 


Lead Author comment (Jack) 


Although | like this concept, in practice it won't be "anyone" that is submitting or bothering to 
standardise data. The people you are ultimately trying to target are researchers, NGOs, governments 
etc., so maybe worth mentioning who the specific target audiences are (including the encouragement 
of citizen scientists). With that in mind, it is worth considering that those people (like me!) will not be 
experts in standardising jargon so it may be good to be as specific as possible (using examples) 
throughout. Also, make it clear how STARDIT will run alongside government and academic outputs. 


Agreed, it says all those carrying out research and interventions, which covers what I say next. It may 
be worth highlighting somewhere in the paper that another really important factor is that it provides 
information to various development aid agencies (governments, NGOs etc.) about what initiatives has 
and are being done, which can save precious time and money for regions that can ill afford to have 
limited aid budgets blown on projects already done elsewhere. 

The three main issues with aid architecture include: poor information sharing, coordination and 
planning; no info on results effectiveness and aid allocation rules that lead to a limited ability to scale 
up, learning and innovation. STARDITs design can help with all these issues! See Page 17 of this 
highlights this issue:https://www.brookings.edu/wp- 
content/uploads/2016/06/11_development_aid_kharas.pdf 


| am not saying that this should be mentioned here, but it might be worth adding as a benefit 
somewhere else? 


| know from my experience in the Pacific, there was so much money being wasted everywhere - with 
aid money being splashed everywhere on projects that often duplicated each other. No one had any 
real idea who was doing what in the Pacific? Often we would hear about a project that was closely 
matched to our own, but could never access documents, info etc. Different countries involved. With 


STARDIT there could be greater information sharing, leading to greater coordination and planning and 


improvements from access to info on outcomes e.g., use lessons learnt and increase the likelihood of 
greater success in aid projects. 
... global air pollution and habitat loss, 


Second 
Author's 
response 
(Thomas) 


Lead author's 
response if no 
change made 
(Jack) 


I think this is a wider 
point about STARDIT 
that is touched on 
later in the 
development phases 
and also a point 
about messaging 
and audience 
targeting, which | 
think is something 
to address for 
version One 


Attempted to work in more examples Also, examples throughout would be really helpful. een Ft eed 


If a project has access to STARDIT would that have any benefits? E.g. lowered costs, improved results? 


Can we put some numbers/values in to add strength to the initiative? 


| think this is a great 
idea, but beyond my 
capacity and 
expertise for the 
Beta version and 
certainly a good 


revised sentence strcuture but kept 'citizen science’ in inverted commas as it requires defining, 
and using it outside commas suggests it has a fixed meaning the reader should understand, 
which often it doesn't. 

clarified wording 


Background 


excellent. Included 


added references to GBIF and references about the limitations of such platforms 


Thank you, noted. Vulnerable has a very specific meaning in certain contexts (for example, 
vulneralbe adults in the mental health context) but whether this term should be applied to whole 
populations is a good question. | would prefer soemthing like ‘at greater risk of exploitation’. 
Minority is problematic, as in what frame of reference is someone in a minority, and who is doing 
that framing? Also, very rich people are in a minority, but are not ‘vulnerable’. Lots of lethal 
euphemisms here! 

added comments about data access and citations. 


Second 
Author's 
response 
(Thomas) 


Lead author's 
response if no 
change made 


Lead Author comment (Jack) 


Why is citizen science in inverted commas? Is it to suggest it is still a developing science, not yet 
established? Not saying it is, the inverted commas infer that. 


Maybe clarify what it means to "involve people": involve people in research? Involve people in action- 
planning? Also, ultimately standardising data saves time/money and results in more accessible 
evidence-based action planning (whether that is by governments or NGOs etc.). 

See my previous comments above. Really like the opening paragraphs, but now it seems STARDIT is 
aimed at ‘governments, industry, research organisations and people around the world’. Needs to be 
clarified earlier who STARDIT is for, and how it will work alongside/in place of existing publication 
platforms. 


the audience is 
‘everyone’, have 
attempted to make 
use cases for 
different audiences 
clearer in the table 
of examlpe use 
cases 

This is covered in 
detail in additional 
resources 


Who will do this? Will it be peer-reviewed? How is quality in the database assured? Will it have a 
Wikipedia model of self-review? 


added reference This is also similar to realist evaluation, good to make this link here or later 


| know the UN and data is mentioned above but seeing that STARDIT is about multiple sectors & 
disciplines and that Climate Change is the biggest threat humanity faces, it might also be worth 
mentioning somewhere an environmental body (i.e., not just health and economics) that also outlines 
the importance that information and knowledge sharing has to overcoming this challenge. This could 
be included further down where you mention the benefits of knowledge sharing. For example, the 
Paris Agreement highlighted the critical role of Information and Knowledge Management to Climate 
Change Adaptation in Article 7 that Parties “strengthen cooperation and enhance action” to share 
information, experience and lessons; strengthen institutional arrangements; strengthen scientific 
knowledge; assist developing countries to identify effective adaptation actions and improve 
effectiveness of adaptation actions (UNFCCC 2015). From: UNFCCC (2015) Paris Agreement. 
FCCC/CP/2015/10/Add.1.1_32. See: 
http://unfccc.int/files/essential_background/convention/application/pdf/english_paris_agreement.pdf 
how about adding something about previous successful initiatives for standardizing data. you could 
also include current trends like GBIF which have a similar rationale and also open data/ open source 
initiatives. 

There has been criticism of this term recently eg. 
https://www.tandfonline.com/doi/abs/10.1080/09581596.2019.1656800?journalCode=ccph20 
Perhaps "minority"? 


1. Could it be appropriate to also add in a sentence about data ownership? For example, which 
stakeholders/projects are more likely to share data? Comment that we need to encourage 
stakeholders (that own the data) and educate on the importance of sharing data to enable others to 
take control over their lives/decisions etc. 


covered later in paper but added ‘with interoperability with other platforms planned for future 
versions.’ 


Current usage 


added more examples with references 


Lead Author comment (Jack) 


2. Agree this would be important to define/describe before the STARDIT part and to touch on 
opportunities for community data ownership, sovereignty 

How could STARDIT continue if Wikidata suddenly disappeared? Suggest describing how STARDIT is 
held on a platform that enables accessible data sharing and that Wikidata is currently used 

1. There is a growing call for scientific journals to make reviewers and editors anonymous comments 
and reviews part of the official scientific record. A few journals are taking it up, but resistance is there 
and uptake is slow, STARDIT could help to build a stronger case for it. See: Polka et al. (2018). Publish 
peer reviews. Nature: https://www.nature.com/articles/d41586-018-06032-w. 


| am all for it to help improve transparency and help with better public perceptions of science. 
Although it is worth factoring in that asking reviewers to consent to being identified to the author has 
no important effect on the quality of the review, the recommendation regarding publication, or the 
time taken to review, but it significantly increased the likelihood of reviewers declining to review. 


See: Van Rooyen, S., Godlee, F., Evans, S., Black, N. and Smith, R., 1999. Effect of open peer review on 
quality of reviews and on reviewers' recommendations: a randomised trial. Bmj, 318(7175), pp.23-27. 
link: 
https://www.bmj.com/content/bmj/318/7175/23.full.pdf?casa_token=y5JgdYfuEIcCAAAAA:RsmsQAgdY 
lUglSnUQ-JIQHFIEVQnJA2yEWHmLIKkK5F-KSyBKrHCavVYKOd9YlgUx6c-WeZltsmo1A 


2. | agree with Roan’s comments. I’m all for publishing the reviewer comments, as this will give much 
better transparency. However, if you don’t make them anonymous you are going to find it very difficult 
to find people to review articles, and people will start to be less critical, especially when the 
community is small. It's already incredibly difficult to find experts with free time to review. 


This is great! Could we mention an example to explain the benefit of STARDIT? E.g. What new 
information was shared though STARDIT that wasn’t published by the journal (assuming this is where it 
was published)? How could this information be useful to future initiatives? 


Lead author's 
response if no 
change made 


I think this is an 
important issue 
(open peer review) 
but one that is 
outside the scope of 
this paper. In other 
words, open peer 
review is already 
happending (and in 
fact is used by the 
Wiki Journals and 
the journal where 
this will be 
submitted) and in 
most cases where 
I've been part of 
open peer review 
process you can 
chose to be 
anonysmous or not. 


Regardless of the 
benefits or cons - 
the point is that 
STARDIT can be used 
to report 
information about 
peer review process 
- it's not in itself an 
open peer review 
process. How the 
editorial process 
works for future 
STARDIT reports is 
also an open 
discussion. 


Comments flagged 
with Thomas for his 
input as he has 
superior expertise in 
this area 


Second 
Author's 
response 
(Thomas) 


Partial 
solutions 
also involve 
embargoing 
comments 
and/or 
reviewer 
identities 


Potential 
applications 


Added this para with references 'In addition, STARDIT could be used to share information which 
makes research more reproducible43,44, improving accessibility to the information required to 
critically appraise research and evidence and thus improving trust in processes such as the 
scientific method45,46, and facilitate an appraisal of different knowledge systems, including 
Indigenous knowledge systems47. 


Added in this sentence: Such data sharing could also improve the translation of trusted, quality 
research and data, by empowering people to both access and appraise relevant data. For 
example, improved access to more standardised information (in multiple languages) about data 
and outcomes, could help to facilitate more informed collaborations between researchers and 
those monitoring and protecting critically endangered species48—50. 


made it clearer that sometimes meanings are vague and that we are therefore defining what 
they mean in this article: While meanings of these terms are often imprecise and can be used 
interchangeably, ‘involvement’ here is distinct from ‘engagement’, which is where which 
information and knowledge about initiatives is shared, for example, with study participants who 
remain passive recipients of interventions.54 


Lead Author comment (Jack) Lead author's Second 
response if no Author's 


change made response 
(Jack) (Thomas) 


This application could be mentioned earlier as a way of describing the benefit of STARDIT, e.g. ‘one of | mentioned in 

the advantages of standardizing data reporting is that comparisons between methods and impacts abstract - repitition 

across multiple initiatives can easily be made.’ if mentioned in 
introduction? 

STARDIT might also help with the scourge of irreproducible scientific studies. Tens of billions of dollars 

are wasted each year on irreproducible research, where insufficient information on methodology and 

data prevents many studies from being reproduced. With 14% of scientists reporting that they have 

witnessed scientific fraud (Fanelli 2009), there is a push for more stringent demands of proof from 

investigators/ scientists (Clark 2017) e.g., video taping of experiments and full raw data. STARDIT could 

facilitate this. Examples of the issue, see: Clark, T.D., 2017. Science, lies and video-taped experiments. 

Nature News, 542(7640), p.139. Link: https://www.nature.com/news/science-lies-and-video-taped- 

experiments-1.21432 AND Fanelli, D., 2009. How many scientists fabricate and falsify research? A 

systematic review and meta-analysis of survey data. PloS one, 4(5), p.e5738. link: 

https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0005738 

there is also an opportunity to improve our overall understanding of social and environmental 

problems by embracing the strengths and weaknesses of the different knowledge systems (UNESCO 

2017) UNESCO, 2017: Local knowledge, global goals. Local and Indigenous Knowledge Systems 

Programme Rep., UNESCO, 48 pp., http://unesdoc.unesco.org/images/0025/002595/259599E. pdf. 

...also help to improve public trust in scientific method and outcomes. e.g., 

https://www.nature.com/articles/nbt1199supp2_14 and 

https://journals.sagepub.com/doi/pdf/10.1177/0963662519869097 

For a multidisciplinary environmental benefit that STARDIT could also help with: As with health and 

other disciplines, there is a big issue with a lack of knowledge translation between scientists and 

conservation managers (see Linklater 2003for an example). With the ongoing biodiversity crisis (Ripple 

et al. 2017), having improved access to more standardised shared project information, data and 

outcomes, could help to facilitate a better understanding of the quality of what work has been done 

and lead to more informed collaborations between managers and researchers, perhaps to more 

applied projects on conservation management focussed research for improved decision making when 

managing critically endangered species. An example of how a lack of standardised info and data can 

lead to poor outcomes, rhino conservation managers had to rely on several non-standardised home 

range study sources for black rhino, incorrectly interpreting the impact of the non-standardised 

studies, and initiated actions that had significant detrimental impacts for a key black rhino population 

(See Plotz et al. 2016). 

References: 

Linklater, W.L., 2003. Science and management in a conservation crisis: a case study with rhinoceros. 

Conservation Biology, 17(4), pp.968-975.Doi: https://doi.org/10.1046/j.1523-1739.2003.01449.x 


Plotz, R.D., Grecian, W.J., Kerley, G.I. and Linklater, W.L., 2016. Standardising home range studies for 
improved management of the critically endangered black rhinoceros. PLoS One, 11(3), p.e0150571. 
Doi: https://doi.org/10.1371/journal.pone.0150571 


William J. Ripple, Christopher Wolf, Thomas M. Newsome, Mauro Galetti, Mohammed Alamgir, Eileen 
Crist, Mahmoud I. Mahmoud, William F. Laurance, 15,364 scientist signatories from 184 countries, 
World Scientists’ Warning to Humanity: A Second Notice, BioScience, Volume 67, Issue 12, December 
2017, Pages 1026-1028, https://doi.org/10.1093/biosci/bix125 

Not sure this is the best example, engagement is surely more than just being a study participant with 
no input or contribution. Agree involvement is a more active term than engagement 


implementation | on the next version, (version 1.0). ' 


Lead Author comment (Jack) 


Added 'see Table X in appendix for examples of data standards which could be incorporated' as 
there's a big list in what is currently table 5 

added 'Development has also been influenced by existing work in health research, including the 
multidisciplinary area of public health, which incorporates social, environmental and economic 

research. ' 


added ref Another example of community ownership and TEK principles in WA: 
https://researchonline.jcu.edu.au/24751/1/ES-2012-5165.pdf 
added ref also relevant is O'Donnell, E.L. and Talbot-Jones, J., 2018. Creating legal rights for rivers. Ecology and 
Society, 23(1). http://www.ecologyandsociety.org/vol23/iss1/art7/; 
https://www.routledge.com/Legal-Rights-for-Rivers-Competition-Collaboration-and-Water- 
Governance/ODonnell/p/book/9780367584160 
| think this is such an important paragraph! Also, the article touches on this, but are we not excluding 
different types of people/cultures already by asking them to fill in these reports? | am just thinking of 
people | work with in Indonesia and China, many of whom, would not respond well to a wordy 
document/guidelines and would need strong benefits to encourage them to use STARDIT (which goes 
back to one of my original points). 


and say that this can be applied across disciplines? Public health is already multidisciplinary, 
incorporating economics, environment, social etc. 


added 'In addition, ongoing co-design will be required to ensure STARDIT is as accessible and 
inclusive as possible. ' 


Lead author's 
response if no 
change made 

(Jack) 


This is a very 
important point, and 
one raised by an 
aboriginal 
community member 
too, it comes down 
to accebility and if 
there's training in 
place. I've 
mentioned this in 
limitations but also, 
frankly, it's a 
limiation of ALL 
kinds of publishing 
and any online tool. 
If anything, being 
able to complete it 
in other langauges 
makes it more 
accecible but 
ongoing co-creation 
is needed to keep 
improving it 
(including training) 


Second 
Author's 
response 
(Thomas) 


Such as? As someone who comes from a very niche background, it would be useful to know what other 
established data standards can/are used. 


added ' Is this the first mention of Science for All? Maybe some more information on who they are, their values 
and objectives? 


changed to 'Once STARDIT Beta (version 0.2) has been submitted for publication, work will begin | Perhaps confidence in the current version should enable version 1.0 work to commence regardless of 
publication status. The journal may not want to publish an interim version 
would be useful to describe the make-up of the working group, in categories e.g. researcher, citizen 


scientists or as a figure? 


| have added 'more detail in appendicies' 


I'm keen to keep it 
brief in the main 
body and also open 
ended (not too 
prescriptive) but | 
have added 'more 
detail in 
appendicies' 


How did Wikipedia become so universally adopted? Any lessons or examples there we can use to 
improve adoption and reporting into STARDIT? 


applications 


Table 1: 
Example 
applications of 
STARDIT 


Thank you - incorporated and added refs 


Lead Author comment (Jack) 


How did Wikipedia become so universally adopted? Any lessons or examples there we can use to 
improve adoption and reporting into STARDIT? 


Agreed. As an fyi, there is a regional information and knowledge management system called iCLIM. It is 
not across any discipline but many of the principles align with STARDIT's objectives. The Pacific iCLIM 
Project aims to enable better climate change resilience and adaptation planning in the Pacific by 
improving the discoverability, storage, access, and utilisation of climate change data and information. 
For more info see here: https://www.griffith.edu.au/research/research-excellence/griffith-climate- 
change-response-program/pacific-iclim and a situation analysis here: https://www.redicomar.com/wp- 
content/uploads/2019/12/Pacific-Situation-Analysis_Pacific-iCLIM-Project-2019.pdf 

The Table is great. In the first two pages of it, there is a lot of repetition of categories that overlap 
across all the different Areas & sub-areas (e.g., people affected/ involved; process for deciding and 
measuring outcomes; experts involved; in most cases 'funding'). Just a suggestion: could all the similar 
categories that occur within and across all the areas/ sub-areas be collated in a separate table to say - 
‘these categories will be recorded as standard across all areas, and then have this table show any 
additional categories that are uniquely/ specifically recorded for each of these different application 
areas. It might make it easier for the reader to understand what is being recorded for their area of 
interest and make more meaningful comparisons. As it is now, | found it a bit hard to absorb all the 
various categories relevant to specific areas with all the repetition. Also, an added benefit once all the 
similar/standard categories across areas are evaluated and collated, allows easier comparisons to 
evaluate whether any of the categories currently listed should actually be recorded for that area or if a 
category currently not considered in area should actually be recorded and vice versa. The added 
complication is that this is an evolving tool and categories that are currently listed within an area might 
shift, and others likely to be added and removed over time - we could add an appropriate caveat to 
indicate that. 

important to engage with Indigenous people around this content, hope this has occurred through 
Poche Centre engagement but if not yet, suggest engaging directly with Boe 


Data custodianship' might cover this but might be worth mentioning whether the initiative followed 
any local, national and international legislation and policies that might be in place to govern the way in 
which TK is collected, documented, and stored. When projects involve TK, there is a need to be aware 
of any legal frameworks that may apply, including those designed to protect cultural and intellectual 
property (IP). This can be in the form of national 

policies and Acts and /or local cultural restrictions. This will vary across the globe. 

In short, have initiatives documenting Traditional and Indigenous Knowledge considered the following 
key issues: 

(1) legal and national contexts; (2) prior informed consent; (3) cultural restrictions; 

and (4) IP rights. For example, the use of prior informed consent is part of a best practice approach. In 
the Pacific, we provided information to participants about the project, including project purpose, who 


Lead author's 
response if no 
change made 
(Jack) 

I think this is outside 
of the scope of the 
paper but the short 
answer from me 
would be 
transparent 
governance and 
editirial processes, 
which we've 
emulated with 
STARDIT 


| think this is a great 
idea but | just didn't 
have the time to do 
this right now. 
Perhaps this could 
be something we 
work on for future 
version when trying 
to communicate it 
to dofferent 
audiences? 


Thank you - yes, 
STARDIT presented 
at multiple Poche 
Centre meetings, 
and feedback invited 
and incorporated 


Second 
Author's 
response 
(Thomas) 


Lead Author comment (Jack) Lead author's Second 
response if no Author's 
change made 


response 
(Thomas) 


was involved, methods of collection and dissemination and consent was asked, including the level of 
sensitivity of the information e.g., low =publicly available, medium = known only to community or 
knowledge holder and can only be shared with requested permission from the knowledge holders; 
High sensitivity (spiritual info/ customary laws) = to remain with knowledge holder and within 
database. 

For specific example see: 

Malsale, P., Sanau, N., Tofaeono, T.I., Kavisi, Z., Willy, A., Mitiepo, R., Lui, S., Chambers, L.E. and Plotz, 
R.D., 2018. Protocols and partnerships for engaging Pacific Island communities in the collection and 
use of traditional climate knowledge. Bulletin of the American Meteorological Society, 99(12), 
pp.2471-2489. DOI: DOI: https://doi.org/10.1175/BAMS-D-17-0163.1 

| think a measure of gender representation/ inclusiveness in initiatives involving IK needs specific 
consideration here? Indigenous Knowledge, and use, of environmental information can vary and are 
often segregated according to gender. In many IK projects and outputs the female voice is absent. It is 
therefore important to consider gender inclusiveness when collecting IK information, particularly as 
the impacts of environmental variability can impact genders differently in many remote and vulnerable 
regions. E.g., Balakrishnan, R., 1998: The Pacific. Rural Women and Food Security: Current Situation 
and Perspectives, Food and Agriculture Organization of the United Nations, 50-60, 
https://www.fao.org/3/W8376E/w8376e05.htm. AND Lane, R., and R. J. G. McNaught, 2009: Building 
gendered approaches to adaptation in the Pacific. Gend. Dev., 17, 67-80, 
https://doi.org/10.1080/13552070802696920. 

There is an example of a database for Traditional Knowledge forecast indicators in the Pacific Islands 
that was designed with such added levels of security to culturally sensitive information, through 
limiting access according to restrictions imposed by the TK expert/ community who initially provided 
the information. 

For info see: Chambers, L.E., Plotz, R.D., Dossis, T., Hiriasia, D.H., Malsale, P., Martin, D.J., Mitiepo, R., 
Tahera, K. and Tofaeono, T.I., 2017. A database for traditional knowledge of weather and climate in the 
Pacific. Meteorological Applications, 24(3), pp.491-502. doi: https://doi.org/10.1002/met.1648 


The TK Database needed to enable restrictions based on things such as: 
user's membership of a clan or tribe; 
user's status/role within the tribe; 


user's gender, and 
the context in which the resource will be reused or reproduced etc. 


at ER G Er ey hea ube eae [lr 
people 


Thank you - incorporated and added refs | have said some of this before in another comment here. e.g., free, prior and informed consent. Also 
worth considering sui generis systems based upon customary law (ref: 
https://mckinneylaw.iu.edu/iiclr/pdf/vol17p67.pdf). For example, this is straight from from UNESCO 
2017 Indigenous Knowledge document: "Many communities are calling for the protection of their 
knowledge from inappropriate use, emphasising the need for free, prior and informed consent and 
benefit sharing. Existing intellectual property regimes are ill-adapted to indigenous knowledge. More 
appropriate methods are being developed, such as sui generis systems based upon customary law. 
There is also knowledge that the community may want to keep for themselves (e.g. locations of sacred 
groves and preferred harvesting areas). It is important to understand the different types of knowledge, 
and both the individual’s and community’s right to control access. " 

UNESCO, 2017: Local knowledge, global goals. Local and Indigenous Knowledge Systems Programme 
Rep., UNESCO, 48 pp., http://unesdoc.unesco.org/images/0025/002595/259599E.pdf®. EditSign 


done - good suggestion Add a referenced definition? e.g. UK MRC guidance? ones? [teal 


Lead author's 
response if no 
change made 
(Jack) 


Mapping 
preferences for 
involvement 


Done - and changed strucuture of discussion Highlight more STARDIT strengths in these early paragraphs e.g. the strength of co-design in its 
development, interdisciplinary nature etc. 

changed to 'estimated' - there's no preceedant really, but GRIPP2 is quite old now and one How or on what basis is 5-years deemed the likely time-frame? Any precedence or examples to 

review says it's not very well used compare -e.g., wikijournal? 


Second 
Author's 
response 
(Thomas) 
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Standardised Data on Initiatives (STARDIT) Beta Version: Additional 
File 1 


About this document 


This document contains additional information relevant to the article ‘Standardised Data on Initiatives 
(STARDIT) Beta Version’, which is currently under review (as of 1* November 2021), and is available at 
this link: https://doi.org/10.21203/rs.3.rs-934285/v1 


A direct link to Additional File 1 can be found here: https://assets.researchsquare.com/files/rs- 
934285/v1/db63b967750ed4f587709416.pdf 


Results 


Chapter 4 — Additional file: Public Involvement in Global Genomics 
Research: A Scoping Review 


Additional file 1: GA4GH data and results 


A direct link to Additional File 1 can be found here: 
https://www.frontiersin.org/articles/file/downloadfile/446268 supplementary- 


materials tables 1 xlsx/octet-stream/Table%201.XLSX/1/446268 


Additional file 2: Systematically searching sites for ‘public involvement’ and 
related concepts 


A direct link to Additional File 2 can be found here: 
https://www.frontiersin.org/articles/file/downloadfile/446268 supplementary- 


materials tables 2 docx/octet-stream/Table%202.docx/4/446268 

Using the search strategy described below, the public domain websites of all the included initiatives in 
the GA4GH database were searched for reports of involvement and any impacts. This document 
describes Stage 1 and Stage 2 of the scoping review in more detail. The graphic below visually 


represents the different phases at each stage of the review. 


This document contains additional data relevant to the case study ‘Involving elderly research participants in the co-design of a future multi-generational cohort study’. Contact Jack.Nunn@Latrobe.edu.au - orcid.org/0000-0003-0316-3254 
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Stage 1 — Defining “involvement” and the search strategy 


Stage 2 — Searching websites (data extraction) 


Phase one - manual search 


Phase two - Adaptive search terms added 


Phase three: systematic site search 


Stage 3 — Defining inclusion/exclusion criteria, data synthesis and analysis 
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Stage 1 — Defining “involvement” and the search strategy 

We conducted a narrative review of systematic reviews to inform our search terms. In addition we used 
similar studies and existing frameworks to inform the word list 2435596559:36,77,78,87,321,328,333,341 We 
developed a criteria to define ‘involvement’ based on the International Association for Public 
Participation’s participation spectrum and other studies?®™8%319.321. The final list was checked by the 


research team after being tested. Below are the search terms included in the main search. 


Main search terms to describe ‘involvement’ 


An asterisk (*) denotes that all possible grammatical variations of the nouns used to describe people 
involved. 
e Involvement (involv*) 


e Engagement (engag*) 
e Partnering (partner*) 


Main search terms to describe people involved 


An asterisk (*) denotes that all possible grammatical variations of the nouns used to describe people 
involved. 

e public* 

e communit* 

e consumer* 

e patient* 

e stakeholder 

e user* 

e citizen* 

e Lay (‘people’ included to exclude phrases such as ‘involved laying down’) 

e Patient 

e PPI (an acronym commonly used in the UK which stands for ‘patient and public involvement’) 


Stage 2 — Searching websites (data extraction) 


Public domain websites of all the initiatives in the GA4GH database were searched for reports of 
involvement and associated impacts. 


Phase one: manual search 


Public domain websites of all initiatives in the GA4GH database were manually searched for reports of 
involvement and associated impacts. 


Phase two: Adaptive search terms added 


During the manual search, adaptive (context dependent) search terms are added to phrase generation 
table. The adaptive search terms which were added to the phrase generation table were: 

e = lay* 

e carer* 

e  volunteer* 

e population* 3?8 

e group* 

e residents (geographical grouping) © 

e participa* (context dependant) 


This document contains additional data relevant to the case study ‘Involving elderly research participants in the co-design of a future multi-generational cohort study’. Contact Jack.Nunn@Latrobe.edu.au - orcid.org/0000-0003-0316-3254 
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e representative 
e payer* 

e taxpayer* 

e customer* 33 
e  client* 333 

e advocate* 

e civil societ* 


When any of these terms returned a result that was within the inclusion criteria, the exact search string 
was recorded. 


Phase three: systematic site search 
Phrase-generation table 


Commercial search engines like Google do not allow an asterisk (*) to denote all variations in 
grammatical endings, so searching for all possible variations must be done manually. Search engines like 
Google do allow the ‘OR’ operator (up to 35 words per search), meaning multiple variations can be 
searched at once. Once the list of words was finalised, the variations were generated in order to 
systematically create search strings. This checklist was used to ensure all possible verb and noun forms 
variations were used. 
Verb forms 

e Base (root) form or infinitive (involve, engage, partner) 


e Active or plural form (involves, engages, partners), used as the present indicative in the third- 
person singular 

e Past tense (involved, engaged, partnered) 

e present participle, gerund, verbal and deverbal nouns (involving, engaging, partnering) 


Nouns 
e Singular (involvement, engagement, partnership) 


e Plural (involvements*, engagements*, partnerships) 


*Not found in prototype searches 


The table below demonstrates this process for systematically combining the various words to describe 


people involved with the terms used to describe involvement. 


This document contains additional data relevant to the case study ‘Involving elderly research participants in the co-design of a future multi-generational cohort study’. Contact Jack.Nunn@Latrobe.edu.au - orcid.org/0000-0003-0316-3254 
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Term for ‘the public’ Involvement Engagement Partnering 
public* “public involvement” “oublic engagement” | “public partner” OR 
OR “public involved” OR | OR “public engaged” | “public partners” OR 
“involving public” OR OR “engaged public” | “public partnership” 
“involves public” OR OR “engaging the OR “public 
“involving the public” public” partnerships” OR 
“public partnering” 
OR “partnering with 
the public” 
“community “community “community 
involvement” OR engagement” OR partner” OR 
communit* “communities involved” | “community “community 
OR “involving engaged” OR partners” OR 
communities” OR “engaged “community 
“involved communities” | community” OR partnership” OR 
OR “involve “community “community 
communities” OR engages” OR partnerships” OR 
“involves the “engaged “community 
community” community” OR partnering” 
“engaging the OR “community 
community” OR partnered” OR 
“community “partnering with the 
engaging” community” 
consumer* “consumer “consumer “consumer partner” 
involvement” OR engagement” OR OR “consumer 
“consumers involved” “consumers engaged” | partners” OR 
OR “involving OR “engaged “consumer 
consumers” OR consumer” OR partnership” OR 
“involved consumers” “engaged consumers” | “consumer 
OR “involve consumers” | OR “consumer partnerships” OR 
OR “involves engages” OR “consumer 
consumers” “engaged consumer” | partnering” 
OR “engaging the OR “consumers 
consumer” OR partnered” OR 
“engaging the “partnering with the 
consumers” OR consumers” 
“consumer engaging” 
patient “patient involvement” “patient “patient partner” OR 
OR “patients involved” engagement” OR “patient partners” 
OR “involving patients” | “patients engaged” OR “patient 
OR “involved patients” OR “engaged patient” | partnership” OR 
OR “involve patients” OR “engaged “patient 
OR “involves patients” patients” OR “patient | partnerships” OR 
engages” OR “patient partnering” 
“engaged patient” OR | OR “patients 
“engaging the partnered” OR 
patient” OR “partnering with the 
“engaging the patients” 
patients” OR 
“patients engaging” 
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Term for ‘the public’ 


Involvement 


Engagement 


Partnering 


stakeholder 


“stakeholder 
involvement” OR 
“stakeholder 
involved” OR 
“involving 
stakeholders” OR 
“involved 
stakeholders” OR 
“involve 
stakeholders” OR 
“involves 
stakeholders” 


“stakeholder 
engagement” OR 
“stakeholders 
engaged” OR 
“engaged 
stakeholder” OR 
“engaged 
stakeholders” OR 
“stakeholders 
engage” OR 
“stakeholder 
engages” OR 
“engaged 
stakeholder” OR 
“engaging the 
stakeholder” OR 
“engaging the 
stakeholder” OR 
“stakeholders 
engaging” 


“stakeholder 
partner” OR 
“stakeholder 
partners” OR 
“stakeholder 
partnership” OR 
“stakeholder 
partnerships” OR 
“stakeholder 
partnering” 

OR “stakeholder 
partnered” OR 
“partnering with the 
stakeholders” 


OR “citizen involved” 
OR “involving citizens” 
OR “involved citizens” 
OR “involve citizens” OR 
“involves citizens” 


OR “citizen engaged” 
OR “engaged citizen” 
OR “engaged citizens’ 
OR “citizens engage” 
OR “citizen engages” 
OR “engaged citizens’ 
OR “engaging the 
citizens” OR 
“engaging the citizen” 
OR “citizen engaging 


) 


? 


user* “user involvement” OR | “user engagement” “user partner” OR 
“user involved” OR OR “user engaged” “user partners” OR 
“involving users” OR OR “engaged user” “user partnership” 
“involved users” OR OR “engaged users” OR “user 
“involve users” OR OR “users engage” partnerships” OR 
“involves users” OR “user engages” “user partnering” 
OR “engaged users” OR “user partnered” 
OR “engaging the OR “partnering with 
users” OR “engaging the users” 
the user” OR “user 
engaging” 
citizen* “citizen involvement” “citizen engagement” | “citizen partner” OR 


“citizen partners” 
OR “citizen 
partnership” OR 
“citizen 
partnerships” OR 
“citizen partnering” 
OR “citizen 
partnered” OR 
“partnering with the 
citizens” 


as ‘involved laying 
down’) 


Lay (‘people’ included 
to exlude phrases such 


“lay involvement” OR 
“ay involved” OR 
“involving lay people” 
OR “involved lay 
people” OR “involve lay 
people” OR “involves 
lay people” 


“lay engagement” OR 
“lay engaged” OR 
“engaged lay” OR 
“engaged lay” OR “lay 
engage” OR “lay 
engages” OR 
“engaged lay” OR 
“engaging the lay” OR 


“lay partner” OR 
“lay partners” OR 
“lay partnership” OR 
“lay partnerships” 
OR “lay partnering” 
OR “lay partnered” 
OR “partnering with 
lay” 
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Term for ‘the public’ 


Involvement 


Engagement 


Partnering 


“engaging the lay” OR 
“lay engaging” 


Public and/& patient 
Patient and& Public 


“PPI involvement” 
included as it is often 
misused 


“public and patient 
involvement” OR 
“patient and public 
involvement” or 
“involving patients and 
the public” OR 
“involving the public 
and patients” OR “ppi 
involvement” 


“public and patient 
engagement” OR 
“patient and public 
engagement” or 
“engaging patients 
and the public” OR 
“engaging the public 
and patients” OR “ppi 
engagement” 


“public and patient 
partnerships” OR 
“patient and public 
partnerships” or 
“partnering with 
patients and the 
public” OR 
“partnering with the 
public and patients” 
OR “ppi 
partnerships” OR 
“opi partnering” 


The words ‘dialogue’, ‘consultation’ and variations of ‘co-production’ are sometimes used to describe 


involving people, so these terms were also searched for. 


Dialogue 


Consult 


co- 


“public dialogue” OR “public 
dialogues” OR “dialogue with 
the public” 


“public consultation” 


OR “co-design” 


“co-production” OR “co- 
produced” OR “co- 
created” OR “co-designed” 


Systematic site search 


Once the manual search was completed, websites were searched using the ‘site search’ function on the 


Google search engine. This method relies on Google servers having carried out a ‘website crawl’, where 


data from the website is indexed °°. While this method cannot be called ‘exhaustive’, it is an 


appropriate sampling technique for this scoping review. In addition, the limitations of the Google ‘site 


search’ function (with regards to Boolean operators and a maximum character limit of 35 characters per 


search) were partially overcome by the ‘phrase generation table’ which was used by authors to manually 


create an exhaustive list of search operators. 


Once the search string was generated in the phrase generation table it was entered into the Google 


search engine. For example, this search string returned 4 results: 


site: www.ukbiobank.ac.uk/ “public involvement” OR “public involved” OR “involving public” OR 


“involves public” OR “involving the public” 


If aterm such as ‘patient’ was commonly used on a site (and thus had more than 10 pages of search 


results) ‘intext’ operators were used to refine the search to only return page results where another 


specific term appears. For example: 


site:www.irdirc.org patient intext:participation 


When search strings returned a result, these were logged in the extraction document. 
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the co-design of a future multi-generational cohort study 


Additional File 1: Data and Analysis 


About this document 


This document contains additional data relevant to the case study ‘Involving elderly research 


participants in the co-design of a future multi-generational cohort study’. In addition it contains a more 


detailed description of the data sources in this case study. This document includes the preferences 
mapping data (STARDIT-PM), and other data about this initiativet. The corresponding Standardised 
Data on Initiatives Alpha Version (STARDIT) of the report can be found in Additional File 5. 


Data mapping 


This table summarises all the data sources used for the case study. 


Data Category 
Diary 
ASPREE Newsletter 


Meeting records 


Email discussions 


Reports on progress 
Interview participant initial 
feedback 

Meeting about interviews 


Interview data 


Interview summary 
Meeting about event 


Interview 

Event planning feedback 

Pre and post event information 
and questions 

Facilitation plan and relevant 
reflections 

Event recording 


Participant feedback about event 


Email summaries of event 


Data point description 

ASPREE research diary — including reflections 

Advert in the newsletter for all participants asking for 
those ‘interested in providing input on the design of 
possible future genetic, family and multi-generational 
studies’ to get in touch 

Meetings with the study team, including notes, audio 
recordings and relevant emails 


Emails about updated versions of the questionnaire May 
2018 and June 2018 

Email discussion with study team members 

Interview reports from MS — June 2018 

Interviews with participants asking for feedback on 
questionnaire design changes 

Discussion with MS and JN based on interviews conducted 
by mid-June 

Interview recordings (audio and PDF notes) 

Interview response highlights identified by MS 

Interview contents are summarised in a spreadsheet by MS 
Study team meeting about event, informed by interviews — 
August 2018 

Email interview with MS about interviews, including early 
identification of themes and learning points 

Feedback from participant advisor on event facilitation 
plan — August 2018 


Information and questions sent to participants before and 
after the event 

The final facilitation plan and relevant reflections in JN’s 
research diary 

Audio 

Video of event 

Short video interviews with 4 event participants 
Participant feedback about the event, including feedback 
forms 

Email summaries about event 
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Data Category 

Notes from event 

Meeting notes 

Email discussions 

Discussion about newsletter 


Final newsletter 


Budget of involvement 


Study team interviews 
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Data point description 

Notes from the event by MS, PL, BH and JN 

Meeting notes from discussion with PL and BH 

Emails to study team after event about survey — mid 
September 

Feedback on newsletter by study team members — late 
September 

Final newsletter sent out to event participants and other 
people interviewed 

Budget documents 


Interviews carried out by email with study team members 
6 months after the event - March 2019 


Advert in Winter/Spring 2017 newsletter 


CALL OUT FOR INPUT ON GENETIC FAMILY AND 
MULTIGENERATIONAL STUDIES 


ASPREE has provided an opportunity to investigate factors affecting health 
and well-being in older people. What is not understood is how factors 
affecting health, such as lifestyle and genes, can run in families and across 


generations. 


Your thoughts on how to best undertake future studies related to the 
genetics, health and well-being of families are most welcome. 


If you are an ASPREE participant (or have a family member) who is 
interested in providing input on the design of possible future genetic, 
family and multi-generational studies, please register your interest by 
email: aspree@monash.edu or ring 1800 728 745 (toll free from a landline). 


Upon receipt of your expression of interest, you may be contacted by a 
member of the Public Health Genomics Program at Monash University, 
who works closely with ASPREE on current genomic sub-studies such as 


the Resilience Project. 


Your contribution to the design of genetic family studies can be as little or 
as much as you wish and you can cease involvement at any time. 


Budget 
The budget of the process is itemised below. 
Item Cost (SAUD) 
Room hire and food for event 1000 
Staff time (estimated) 9000 
Total 10000 
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Study team survey 


The following questions were emailed to the study team members six months after the event: 


10. 


11. 


Please describe your tasks in the process of involving people in planning of the new ASPREE 
multi-generational study 


What did you learn from the process of involving ASPREE participants in the research 
planning phase? 


Please describe specifically what worked well or was useful about the way people were 
involved 


Please describe specifically what did not work well or was not useful about the way people 
were involved 


Were there any barriers or facilitators to conducting the involvement activities? 
(institutional or otherwise) 


Do you think the involvement activity achieved its intended aim(s)? 


Do you have any advice to other researchers planning participant involvement for their 
research? 


Describe the impact you think involving people had (positive/negative - on the research, 
staff or participants) 


Who do you think should influence the kind of human genomic research done in the future, 
and why? (e.g. the public, participants of research studies, doctors, school children, 
politicians etc) 


Which stages of future genomic research should be influenced by people other than 
researchers (if any)? (e.g. concept planning of new studies, study design, conducting the 
research, presenting the results etc) 


Other comments 
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Area (quantitative data about 
responses in this area) 


Qualitative summary of participants’ views and 
perspectives 


Qualitative summary of study team’s views and 
perspectives 


Views on who should be involved: 


50% (10/20) of interview participants, 
100% (18/18) of event participants 
and three (75%, 3/4) study team 
members shared a view or perspective 
about this area. 


Two participants noted that the purpose of involving 
people needed to be clear in order to avoid ‘wasting time’. 


One participant felt only researchers should be involved as 
they are ‘the qualified people’, two others stated 
participants should be involved as researchers only ‘see it 
from their point of view and nobody else’s’, and 
participants bring ‘new perspectives’. 


One participant mentioned ‘vested interests’ and 
suggested involving participants was a way of overcoming 
this. 


Transparency from the project about who is involved 
(specific professions) might help participants identify 
‘different directions’. 


Event participants were unanimous that there was no 
aspect of the research that they should not be involved in. 
One participant stated that ‘funding’ decisions may be 
better being made by experts, although participants 
agreed they should be involved in the oversight of research 
funding. 


One study team member (an ASPREE participant 
assessor) reflected that the ‘increased autonomy’ 
of involving other staff equally (rather than just 
senior research staff) made them feel valued and 
gave the opportunity to ‘think creatively’ and 
‘engage in controversial or difficult discussions’. 


Views on who should do which tasks: 


Participants stated they should be involved in research 
design. A participant commented that feedback is needed 
from participants. Another participant suggested that 


The lead investigator stated that participant 
involvement “significantly improves the 
researchers’ ability to make sound decisions 
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Area (quantitative data about 
responses in this area) 


Qualitative summary of participants’ views and 
perspectives 


Qualitative summary of study team’s views and 
perspectives 


25% (5/20) of interview participants, 
100% (18/18) of event participants 
and two (50%, 2/4) study team 
members shared a view or perspective 
about this area. 


participant information can be confusing and that a 
layperson can have the task of simplifying it. A third 
participant expressed a willingness to be involved as long 
as the task had purpose and was not ‘just for the sake of 
chatting’. 


10 event participants said they would be interested in 
being involved in recruitment and communication, 7 were 
willing to be involved in data access decisions and 2 in 
ethical decisions. 


regarding the fundamental research questions, 
study design, ethics and funding applications”. 


The participant advisor stated ,‘I believe that 
researchers should have the dominant 
responsibility to plan and complete genomic 
research’. 


Views on modes of communication: 


65% (13/20) of interview participants, 
100% (18/18) of event participants 
and 25% (1/4) of study team members 
shared a view or perspective about 
this area. 


There was much variation in views and perspectives about 
communication mode. For example, some participants 
stated a preference for face-to-face discussion, while 
others preferred online questionnaires, commenting on 
documents online or joining online text-based discussion 
groups. 


Participants reported perceived advantages and 
disadvantages for each communication mode. Two thirds 
of event participants said that they would be happy to be 
involved both face-to-face and online (using computers 
and smartphones). 


Event participants felt certain tasks (such as reviewing 
information) could be done ‘more online’, and that face-to- 
face meetings were helpful when there was an ‘occasional 
need’. Online text-based discussions were stated to have 
advantages by ‘opening up more discussion’ as it gave a 
chance for people to reflect on other participants’ views 
and perspectives, meaning discussion could be more in- 
depth. 


Referring to the face-to-face event, one study team 
member stated, ‘participants really enjoyed the 
opportunity to be heard and put their views 
forward’. After the event, the lead investigator 
noted that the planned research ‘must use 
mobile/internet technology’. 


This document contains additional data relevant to the case study ‘involving elderly research participants in the co-design of a future multi-generational cohort study’. Contact Jack.Nunn@Latrobe.edu.au - orcid.org/0000-0003-0316-3254 


381 


Area (quantitative data about 
responses in this area) 


Qualitative summary of participants’ views and 
perspectives 


Qualitative summary of study team’s views and 
perspectives 


Views on what methods should be 
used: 


15% (3/20) of interview participants, 
100% (18/18) of event participants 
and all (100%, 4/4) study team 
members shared a view or perspective 
about this area. 


One participant stated that they did not feel comfortable 
being part of a face-to-face group, while another described 
a method of being involved which was a small group 
conversation with a researcher leading a discussion to 
gather views and ideas. 


Another participant suggested having information sent out 
which could be read, with participants providing feedback. 


Event participants spontaneously suggested using an 
online discussion platform and shared views on what is 
good moderation and the advantages of online discussion, 
although some shared concerns about for-profit social 
media platforms being used. 


After the face-to-face event, most of the study 
team members felt that dividing the event 
discussion into small groups facilitated discussion 
and gave more people a chance to share views and 
perspectives. One study team member felt that 
asking focussed questions and requesting a show 
of hands was a time-efficient way to gauge 
perspectives. 


Views on facilitators of involvement: 


15% (3/20) of interview participants, 
100% (18/18) of event participants 
and 100% (4/4) of study team 
members shared a view or perspective 
about this area. 


Giving people early notice of events and clear advice about 
the purpose and expectations were identified as important 
by two participants. 


One participant identified ‘personality’ as distinct from 
skills and knowledge — which could be considered a 
facilitator if managed appropriately. 


100% of event participants felt that if they were involved in 
recruiting participants from their immediate family, a short 
explanatory video would be helpful and improve their 
confidence in explaining the study. 


When asked about support, two event participants 
identified it as helpful having a person act as an 
independent facilitator when involved in working in groups 


One study team member felt education was 
essential, and that assuming a limited knowledge 
of a subject and explaining the basic concepts at 
the start of the event was important, as this 
appeared to support people to make informed 
decisions when contributing to group discussions. 


Another study team member stated that adequate 
funding for involvement was required so that it can 
become ‘a requirement, rather than a luxury’. 


Having a lead investigator ‘who valued the unique 
experiences of each team member and 
participants’ was identified as a facilitator by one 
study team member. 
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Area (quantitative data about 
responses in this area) 


Qualitative summary of participants’ views and 
perspectives 


Qualitative summary of study team’s views and 
perspectives 


on tasks such as ethical oversight (either face-to-face or 
online). 


Views on barriers of involvement: 


40% (8/20) of interview participants 
and 50% (2/4) of the study team 
shared a view or perspective about 
this area. 


Living in rural areas and other travel logistics were 
considered a barrier to participation in face-to-face events 
by a number of participants. 


A lack of clarity about expected time-commitments or 
timing of events was identified by four participants. 


Not having the skills or knowledge was identified as a 
barrier. One participant felt they lacked literacy in using 
computers and online tools. 


One study team member observed after the event 
that they perceived participants from ‘professional 
backgrounds’ dominating group discussions, which 
may have inhibited others. 


Another study team member stated the ‘cost’ of 
involvement in terms of time and financial 
commitment might be a barrier for some research 
projects. 


Views on what the outcome or 
output of the research or 
involvement in research could be: 


15% (3/20) of interview participants, 
22% (4/18) event participants and 
100% of study team members (4/4) 
shared a view or perspective about 
this area. 


Participants raised the issue of wanting to know outcomes 
and outputs of involvement, with one seeking clarity on 
what the purpose of involvement was. 


Participants shared many views about the outcomes of 
research and felt being involved in clarifying the aims of 
the future study was important. Involving participants in 
helping answer what the research ‘hoped to achieve’ was 
an outcome identified by one participant. It was stated 
that ‘responses from participants could cause the experts 
to ask new questions’ or lead the research in ‘different 
directions’. 


One event participant said that her preference was not to 
participate in research which was ‘about the 
aggrandisement of the professor’, while another stated 


All members of the study team thought the 
involvement process achieved the intended aims, 
and that process had a positive impact. None 
reported negative impacts. 


One study team member stated that involving 
participants could help researchers make decisions 
about ‘fundamental research questions, study 
design, ethics and funding applications’. 
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Area (quantitative data about 
responses in this area) 


Qualitative summary of participants’ views and 
perspectives 


Qualitative summary of study team’s views and 
perspectives 


that the most important thing to ‘get right’ was having 
clear study aims with regard to what it is trying to achieve. 
Interview participants suggested that their motivation for 
participating was altruistic, in the hope that the research 
would contribute to positive outcomes for future 
generations. 


Views on which stage of the research 
people should be involved: 


10% (2/20) of interview participants, 
100% (18/18) of event participants 
and 100% (4/4) of study team 
members shared a view or perspective 
about this area. 


Most participants agreed that people other than 
researchers should be involved in research design, 
including designing the research question. A participant 
commented that feedback is needed from participants. 


One study team member stated that ‘participant 
involvement is vital, especially in the early stages’ 
of research, with ‘less involvement’ needed in 
executing the study, collecting data and analysing 
results. Participants could then be more involved in 
the ‘publishing and communication’ of results. 


Research data: 


0% (0/20) of interview participants, 
100% of event participants (18/18) 
and 25% of study team members (1/4) 
shared a view or perspective about 
this area. 


Seven event participants said they would be interested in 
being involved in decisions about data access. 100% of 
event participants were comfortable with their data being 
held by academics. 100% were not comfortable with it 
being held by a for-profit company, although one 
participant said not to ‘rule private companies out 
completely’. All event participants were interested in 
having pharmacogenomic results returned. Two thirds of 
event participants wanted access to their own genomic 
data, and had mixed views about who else should have 
access. GPs were generally trusted to access and interpret 
genomic data, but participants felt GPs shouldn’t have 
access to data that they did not. All but 2 participants 
agreed they should have access to their own data, with 
those disagreeing mentioning cognitive decline as a reason 
for a co-managed model. Some participants had concerns 
about them or their relatives (especially offspring) finding 


During the event, the lead investigator noted that 
while participants overwhelmingly wanted ‘a self- 
managed future of health information’, this was ‘at 
odds’ with the current healthcare professional 
managed information paradigm. 
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Area (quantitative data about 
responses in this area) 


Qualitative summary of participants’ views and 
perspectives 


Qualitative summary of study team’s views and 
perspectives 


out information they ‘might not want to know’. Questions 
about duty of disclosure and how this might affect 
‘employability’ were also asked, with participants seeking 
clarity about how these issues would affect the research 
design regarding return of data. 
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Additional File 2: Telephone Questionnaire: Version Comparison - A comparison 
of the versions of the telephone questionnaire used 


This file is available at this DOI: https://doi.org/10.1186/s40900-021-00271-4 


This is a direct link to the file: https://static-content.springer.com/esm/art%3A10.1186%2Fs40900-021- 


00271-4/MediaObjects/40900 2021 271 MOESM2_ ESM.xlsx 


Additional File 3: Quantitative results - A summary of the quantitative results 
This file is available at this DOI: https://doi.org/10.1186/s40900-021-00271-4 


This is a direct link to the file: https://static-content.springer.com/esm/art%3A10.1186%2Fs40900-021- 


00271-4/MediaObjects/40900 2021 271 MOESM3 _ESM.xlsx 
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Additional File 4 - GRIPP2 report for ‘Involving elderly research participants in 
the co-design of a future multi-generational cohort study’ 


This report has been completed using the ‘GRIPP2 reporting checklists: tools to improve reporting of 
patient and public involvement in research’ available at: https://doi.org/10.1136/bmj.j3453. 


GRIPP2 short form 
Category Category description 
la: Aim Report the aim of the study 


1b: Methods 


1c: Results 


1d: Conclusions 


1e: Keywords 


This document contains additional data relevant to th 


Describe the methods used 
by which patients and the 
public were involved 


Report the impacts and 
outcomes of PPI in the study 


Summarise the main 
conclusions of the study 


Include PPI, “patient and 
public involvement,” or 
alternative terms as keywords 


je case study ‘Involving elderly research participants in the co-design of a future multi-generational 


Data 


Participatory action research to involve elderly research participants in the co-design of a proposed 
multi-generational cohort study, in order to improve research design, relevance, acceptability and 
recruitment. 


Participatory action research to involve elderly research participants in the co-design of a proposed 
multi-generational cohort study. 


Improved participant information resources, improved wording that is culturally appropriate, improved 
question design for interviews, improved learning resources for participants, improved co-design 
process. 


Involving participants in co-designing a proposed study resulted in changes to the design of the 
proposed study 

The process of involving people can be viewed as a learning experience for both the participants 
involved and study team members. The process changed participant and study team members’ views 
about the value of involvement, which can be viewed as an impact of ‘transformative learning’. 


Public Health; Epidemiology; Preventive Medicine; Medical Ethics; Medical Education & Training; public 
involvement; participatory research; genomics; patient involvement; 
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Category Category description Data 


2a: Definition Report the definition of PPI The words ‘involvement’ or ‘being involved’ describe the concept of people being ‘involved’ in research. 
used in the study and how it ‘This is when research is carried out ‘with’ people rather than ‘on’ them. ‘Involvement’ can also be 
links to comparable studies defined as when other people aside from the research team, such as the public, patients, research 
participants and other stakeholders, actively contribute to the research process. It is the ‘active 
involvement’ in shaping and guiding research, rather than only providing data. 


2b: Theoretical Report the theoretical The process was guided by a number of international participatory action research methodology 
underpinnings rationale and any theoretical frameworks, including the International Collaboration for Participatory Health Research and INVOLVE 
influences relating to PPI in guidance on co-design. An Alpha version of ‘Standardised Data on Initiatives’ (STARDIT) was used to map 
the study people’s preferences for involvement in a standardised way, including mapping views on who should be 


involved and how. STARDIT was then used to guide co-design of the process, and to subsequently report 
how people were involved, using standardised data. 


2c: Concepts and | Report any conceptual or We used a case study research methodology to record and describe the process of involving participants 
theory development | theoretical models, or in the co-design. The process was guided by a number of international participatory action research 
influences, used in the study (methodology frameworks, including the International Collaboration for Participatory Health Research 
and INVOLVE guidance on co-design. An Alpha version of ‘Standardised Data on Initiatives’ (STARDIT) 
was used to map people’s preferences for involvement in a standardised way, including mapping views 
on who should be involved and how. STARDIT was then used to guide co-design of the process, and to 
subsequently report how people were involved, using standardised data. 


3: Aim Report the aim of the study (Participatory action research to involve elderly research participants in the co-design of a proposed 
multi-generational cohort study, in order to improve research design, relevance, acceptability and 
recruitment. 


4a: Design Provide a clear description of [The study team held four meetings to co-design the involvement activities. One participant advisor was 
methods by which patients involved in a number of tasks including reviewing and improving the written information, telephone 
and the public were involved (interview questions, and the facilitation plan for the event. 
After the recruitment and consent process, participants were interviewed by telephone. Participants 
were asked about their willingness to provide feedback throughout the study, and to be involved in 
study design, as well as preferences for modes of communication. 
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Category 


Ab: People 
involved 


Ac: Stages of 
involvement 


Category description 


Provide a description of 
patients, carers, and the 
public involved with the PPI 
activity in the study 


Report on how PPI is used at 
different stages of the study 
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Data 


Eighteen participants attended a four-hour workshop event in central Melbourne. The event was co- 
designed by the study team, and was informed by interview data and international best-practices for 
involvement events. 


3 academic research investigators 
An ASPREE participant assessor 
An ASPREE-XT participant 


Stage 1: Planning 


The study team held four meetings to co-design the involvement activities. One participant advisor was 
involved in a number of tasks including reviewing and improving the written information, telephone 
interview questions, and the facilitation plan for the event. 


Stage 2: Recruitment and telephone interviews 


An advert was placed in a newsletter to 14,268 ASPREE participants. After the recruitment and consent 
process, participants were interviewed by telephone. Participants were asked about their willingness to 
provide feedback throughout the study, and to be involved in study design, as well as preferences for 
modes of communication. The definition of involvement below used in the script was co-designed with 
participants for subsequent interviewees. 


Stage 3: Event 

Eighteen participants attended a four-hour workshop event in central Melbourne. The event was co- 
designed by the study team, and was informed by interview data and international best-practices for 
involvement events. 


The event included an introduction to the proposed MGRS by the lead ASPREE-XT genomics researcher 
(PL); a plain-English introduction to genomics by an expert in genomics who is also an ASPREE-XT 
participant (BH); a summary of the telephone interview results by the interviewer (MS); and an 
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Category Category description Data 


interactive session which included open questions about the types of information participants would 
like returned and recruitment of family members. 

The final session included a presentation and interactive discussion about involvement in research, led 
by the event facilitator (Jack Nunn). This session explored preferences about how people would like to 
be involved, with open and closed questions. Questions included preferences about tasks and modes of 
communication. 


Throughout the event, participants shared their views on a range of issues through interactive 
discussions, voting (by show of hands) and anonymous written feedback. 

Stage 4: Evaluation and analysis 

Members of the study team were surveyed six months after the face-to-face event in order to integrate 
the valuable views and perspectives of those involved in co-designing and delivering the process. Design 
of surveys was informed by frameworks for planning and reporting public involvement (GRIPP2 and 
PiiAF). The study team were asked 11 questions and the data from the four interviews was coded and 
categorised using Standardised Data on Initiatives (STARDIT). 


The stages of qualitative data analysis included data mapping and familiarisation; transcription; coding; 
searching for themes; reviewing themes with study team members (including a participant 
representative); labelling and summarising themes; and reporting the findings. In order to enhance 
validity of the analysis, two authors independently analysed the data thematically, which was then 
checked by a third author (triangulation). Standardised categories (STARDIT) were used during content 
analysis of the data in order to facilitate comparison with other research projects. More information 
about the data sources and a STARDIT report available. 


Ad: Level or Report the level or nature of (Participants were involved at every level of every stage, with more information in section 4C. Everyone 
nature of PPI used at various stages of __ listed in 4B was involved in co-designing every stage of the process. This included refining wording of 
involvement the study participant information, sharing views and advice about the process, proof-reading documents, 


providing feedback on questionnaires, analysing data, informing planning, presenting information to 
participants, interpreting data, and participating in email surveys. 
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Category 


5a: Qualitative 
evidence of impact 


5b: Quantitative 
evidence of impact 


5c: Robustness of 
measure 


6: Economic 
assessment 


7a: Outcomes of 
PPI 


This document contains additional data relevant to the case study ‘involving elderly research participants in the co-design of a future multi-generational 


Category description 


If applicable, report the 
methods used to qualitatively 
explore the impact of PPI in 


the study 


If applicable, report the 


methods used to 


quantitatively measure or 
assess the impact of PPI 


If applicable, report the rigour 
of the method used to 
capture or measure the 


impact of PPI 


If applicable, report the 
method used for an economic 


assessment of PPI 


Report the results of PPI in 
the study, including both 
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Data 


The stages of qualitative data analysis included data mapping and familiarisation; transcription; coding; 
searching for themes; reviewing themes with study team members; labelling and summarising themes; 
and reporting the findings. In order to enhance validity of the analysis, two authors independently 
analysed the data thematically, which was then checked by a third author (triangulation). Standardised 
categories (STARDIT) were used during content analysis of the data in order to facilitate comparison 
with other research projects. 


Involving stakeholders in the co-design process impacted the study in seven specific impacts ways. By 
asking for participants’ views on aspects of the proposed study design, the study team gained insight 

into participant preferences and opinions. While there was diversity in views, the process allowed the 
study team to improve aspects of the study design. 


Twenty relevant interviews were transcribed, coded and categorised, with relevant interviews identified 
by two investigators independently. To reduce any unconscious selection bias, a sample of over 10% of 
the interviews was selected at random. 


We used the ‘Standardised Data on Initiatives (STARDIT)’ Alpha Version to plan and report how 
participant involvement activities positively impacted the study design. STARDIT includes a tool to map 
people’s preferences for involvement in a standardised way, including mapping views on who should be 
involved and how. STARDIT was then used to guide co-design of the process, and to subsequently report 
how people were involved, using standardised data. An Alpha version of the STARDIT framework was 
also used in parallel with the thematic analysis to organise data into pre-defined ‘super-categories’ 
which allow consistent comparison with other data using this reporting framework. 


The entire process of involving people was estimated to cost $10,000 AUD, including staff time, catering 
and event venue hire. The value of the process was summarised by the lead investigator who stated “| 
learnt a lot from the process and am very glad we made the effort”. 


Improved participant information resources, improved wording that is culturally appropriate, improved 
question design for interviews, improved learning resources for participants, improved co-design 
process. 
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Category Category description Data 


positive and negative 


outcomes 

7b: Impacts of PPI | Report the positive and Involving stakeholders in the co-design process impacted the study in seven specific impacts ways. By 
negative impacts that PPl has (asking for participants’ views on aspects of the proposed study design, the study team gained insight 
had on the research, the into participant preferences and opinions. While there was diversity in views, the process allowed the 


individuals involved (including study team to improve aspects of the study design. 

patients and researchers), 

and wider impacts 1: Recruitment and sample collection 
Recruitment and consent for the MGRS will occur online wherever possible, and salvia samples (rather 
than blood) will sent by post to be used as biospecimens for DNA analysis. 


2: Participant communication 
A short video and ‘information pack’, which will explain the MGRS study, will be created to assist with 
recruiting family members. 


3: Participant involvement in governance 
Participants will be invited to be involved in overseeing governance, including funding decisions. 


4: Data access 
Study participants should be involved in controlling data access decisions and policies. 


5: Communication and ways of involving participants 

Participants will be included on study advisory groups, including for study recruitment and 
communication, data access and ethical oversight using multiple communication modes. 

6: Provide feedback to participants about the research 

Participants will be informed about the impact of the research, and how their involvement has affected 


the design and management of the study. 


7: Create learning and development opportunities 
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Category Category description 


7c: Context of PPI | Report the influence of any 
contextual factors that 
enabled or hindered the 
process or impact of PPI 


7d: Process of PPI | Report the influence of any 
process factors, that enabled 
or hindered the impact of PPI 
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Data 


Learning and development opportunities will be created for potential participants, researchers and 
other stakeholders. 


The process took longer than expected. There is confusion over what ethics approval is required in order 
to involve people, especially people who are participants in an ongoing study. 


Involving field staff (as well as senior researchers and academics) provided a valuable perspective, as 
some staff knew some participants personally and had knowledge that senior research staff did not. 


Some study team members worried about over-burdening participants by asking them to do too much, 
however this concern did not seem to be backed up by the data collected, and may be considered a 
barrier to involvement. 


Enablers of involvement - Giving people time to read resources. Clear communication about the 
intention of involving people. 


Barriers of involvement - Face-to-face meetings were difficult to organise. Some participants were 
elderly or lived in remote areas, so face-to-face meetings needed to be minimised where possible. 


The process took longer than expected. There is confusion over what ethics approval is required in order 
to involve people, especially people who are participants in an ongoing study. 


Involving field staff (as well as senior researchers and academics) provided a valuable perspective, as 
some staff knew some participants personally and had knowledge that senior research staff did not. 


Some study team members worried about over-burdening participants by asking them to do too much, 


however this concern did not seem to be backed up by the data collected, and may be considered a 
barrier to involvement. 


| cohort study’. Contact Jack.Nunn@Latrobe.edu.au - orcid.org/0000-0003-0316-3254 


Category 


7ei: Theory 
development 


7eii: Theory 
development 


7f: Measurement 


7 g: Economic 
assessment 


Category description 


Report any conceptual or 
theoretical development in 
PPI that have emerged 


Report evaluation of 
theoretical models, if any 


If applicable, report all 
aspects of instrument 
development and testing (eg, 
validity, reliability, feasibility, 
acceptability, responsiveness, 
interpretability, 
appropriateness, precision) 


Report any information on 
the costs or benefit of PPI 
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Data 


The effective involvement of ‘stakeholders’ also includes involving all relevant staff and health 
professionals at all levels of an initiative, who may have important knowledge or perspectives that 
senior research staff do not. In other words, the ‘PPI’ label for this question does not incorporate all 
relevant stakeholders who are not described by this acronym. 

The participatory action research method gave insights into participants’ preferences that measurably 
impacted on the proposed study design. The improvement of the interview design using the co-design 
process illustrates the value of a flexible and iterative approach to involvement in a study. 

By asking participants their preferences, the study team gained useful insights to inform the design of 
the proposed study. Participants preference for being involved in decision making about funding 
sources, data management and ownership, and what information to share with participants will help 
ensure any future study design aligns with participants’ values, ensuring the design is culturally safe and 
culturally competent 


The ‘transformative learning’ during the process reported from both study participants and the study 
team was an important impact captured by the participatory action research (PAR) method. The process 
showed that it was valuable to create regular involvement opportunities for each stakeholder. Reporting 
this process in a standardised way using ‘Standardised Data on Initiatives’ (STARDIT) meant that impacts 
such as transformative learning could be reported and that this case study can be compared to similar 
studies in the future. 


‘Standardised Data on Initiatives’ (STARDIT)was used to map people’s preferences for involvement ina 
standardised way, including mapping views on who should be involved and how. STARDIT was then used 
to guide co-design of the process, and to subsequently report how people were involved, using 
standardised data. An Alpha version of the STARDIT framework was also used in parallel with the 
thematic analysis to organise data into pre-defined ‘super-categories’ which allow consistent 
comparison with other data using this reporting framework. 


The entire process of involving people was estimated to cost $10,000 AUD, including staff time, catering 
and event venue hire. The value of the process was summarised by the lead investigator who stated “| 
learnt a lot from the process and am very glad we made the effort”. 
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Category Category description 


8a: Outcomes Comment on how PPI 
influenced the study overall. 
Describe positive and 
negative effects 


8b: Impacts Comment on the different 
impacts of PPI identified in 
this study and how they 
contribute to new knowledge 
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Data 


No negative impacts were reported from any participants or study team members at any stage of the 
process. Involving stakeholders in the co-design process impacted the study in seven specific impacts 
ways. By asking for participants’ views on aspects of the proposed study design, the study team gained 
insight into participant preferences and opinions. While there was diversity in views, the process 
allowed the study team to improve aspects of the study design. Outcomes included Improved 
participant information resources, improved wording that is culturally appropriate, improved question 
design for interviews, improved learning resources for participants, improved co-design process. 


Involving stakeholders in the co-design process impacted the study in seven specific impacts ways. By 
asking for participants’ views on aspects of the proposed study design, the study team gained insight 
into participant preferences and opinions. While there was diversity in views, the process allowed the 
study team to improve aspects of the study design. 


1: Recruitment and sample collection 
Recruitment and consent for the MGRS will occur online wherever possible, and salvia samples (rather 
than blood) will sent by post to be used as biospecimens for DNA analysis. 


2: Participant communication 
A short video and ‘information pack’, which will explain the MGRS study, will be created to assist with 
recruiting family members. 


3: Participant involvement in governance 
Participants will be invited to be involved in overseeing governance, including funding decisions. 


4: Data access 
Study participants should be involved in controlling data access decisions and policies. 


5: Communication and ways of involving participants 
Participants will be included on study advisory groups, including for study recruitment and 
communication, data access and ethical oversight using multiple communication modes. 
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Category 


8c: Definition 


8d: Theoretical 
underpinnings 


8e: Context 


Category description 


Comment on the definition of 
PPI used (reported in the 
Background section) and 
whether or not you would 
suggest any changes 


Comment on any way your 
study adds to the theoretical 
development of PPI 


Comment on how context 
factors influenced PPI in the 
study 
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Data 


6: Provide feedback to participants about the research 
Participants will be informed about the impact of the research, and how their involvement has affected 
the design and management of the study. 


7: Create learning and development opportunities 
Learning and development opportunities will be created for potential participants, researchers and 
other stakeholders. 


The acronym ‘patient public involvement’ here is limiting, as it does not incorporate research 
participants and other stakeholders such as study staff, who might have unique insights into study 
design. The words ‘involvement’ or ‘being involved’ describe the concept of people being ‘involved’ in 
research. This is when research is carried out ‘with’ people rather than ‘on’ them. ‘Involvement’ can also 
be defined as when other people aside from the research team, such as the public, patients, research 
participants and other stakeholders, actively contribute to the research process. It is the ‘active 
involvement’ in shaping and guiding research, rather than only providing data. 


An Alpha version of ‘Standardised Data on Initiatives’ (STARDIT) was used to map people’s preferences 
for involvement in a standardised way, including mapping views on who should be involved and how. 
STARDIT was then used to guide co-design of the process, and to subsequently report how people were 
involved, using standardised data. STARDIT includes a tool to map people’s preferences for involvement 
in a standardised way, including mapping views on who should be involved and how. An Alpha version 
of the STARDIT framework was also used in parallel with the thematic analysis to organise data into pre- 
defined ‘super-categories’ which allow consistent comparison with other data using this reporting 
framework. Creating consistency in terminology to describe the ‘who’, ‘what’ and ‘how’ of involvement 
allows better comparison. 


The process took longer than expected. There is confusion over what ethics approval is required in order 
to involve people, especially people who are participants in an ongoing study. 


Involving field staff (as well as senior researchers and academics) provided a valuable perspective, as 
some staff knew some participants personally and had knowledge that senior research staff did not. 
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Category 


8f: Process 


8 g: Measurement 
and capture of PPI 
impact 


8 h: Economic 
assessment 


8i: 
Reflections/critical 
perspective 


Category description 


Comment on how process 
factors influenced PPI in the 
study 


If applicable, comment on 
how well PPI impact was 
evaluated or measured in the 
study 


If applicable, discuss any 
aspects of the economic cost 
or benefit of PPI, particularly 
any suggestions for future 
economic modelling. 


Comment critically on the 
study, reflecting on the things 
that went well and those that 
did not, so that others can 
learn from this study 
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Data 


Some study team members worried about over-burdening participants by asking them to do too much, 
however this concern did not seem to be backed up by the data collected, and may be considered a 
barrier to involvement. 


During the process, both participants and study team members reported changed views about the value 
of involvement in research, demonstrating ‘transformative learning’ and co-construction of knowledge. 


Detailed data was collected and shared. Participants were supportive about being involved, with all 
participants supportive of being involved by providing feedback throughout the research process (100%, 
32/32), with a typical participant response being ‘I’d be happy to be involved’. Views about enablers 
were shared in three of the 20 interviews coded, by all 18 of the event participants and all study team 
members surveyed. Views about barriers were shared in eight of the interviews coded and by half of the 
study team surveys. Mapping of preferences for involvement was completed using the STARDIT-PM 
tool, with the involvement reported using STARDIT Alpha. 


Economic assessment needs to be widened to ‘assessing value’, with one way of this being measured 
being financial. The entire process of involving people was estimated to cost $10,000 AUD, including 
staff time, catering and event venue hire. The value of the process was summarised by the lead 
investigator who stated “I learnt a lot from the process and am very glad we made the effort”. 


A number of significant learning points were identified by the study team when responding to the 
question “do you have any advice to other researchers planning involvement for their research”. 
Significant learning points are were: 

1. Fund and prioritise involvement, make it a requirement 

2. Ethics processes take time, but can improve plans 

3. Know your audience — don’t make assumptions 

4. Value diversity in experience and knowledge 

5. A supportive team improves the experience for all 
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Additional File 5 - Standardised Data on Initiatives (STARDIT) report: ‘Involving 
elderly research participants in the co-design of a future multi-generational 


cohort study’ 
About this report 


This report uses the Standardised Data on Initiatives Alpha version (STARDIT)?”. An Alpha version of this 
STARDIT report is also hosted online in machine readable format.°** 


STARDIT Report Alpha Version 


Identifying information 


Initiative name 


Involving ASPREE-XT participants in co-design of 
a future multi-generational cohort study 


Geographic location or scope 


Australia 


Date range (planned start and end dates of initiative) 


2017-2019 


Purpose of the initiative 


Participatory action research to involve elderly 
research participants in the co-design of a 
proposed multi-generational cohort study, in 
order to improve research design, relevance, 
acceptability and recruitment. 


Organisations or other initiatives involved (list all if 
multi-centre) 


1. Department of Epidemiology and 
Preventive Medicine, School of Public 
Health and Preventive Medicine, 
Monash University 

2. School of Psychology and Public Health, 
La Trobe University 


Funding sources 


Department of Epidemiology and Preventive 
Medicine, School of Public Health and Preventive 
Medicine, Monash University 


Clinical trial registration details (if applicable) 


https://clinicaltrials.gov/show/NCT01038583 


Ethics approval (if applicable) 


Monash University 


Other relevant information (free text) 


This report describes involving potential 
participants in co-designing a proposed multi- 
generational research study. It would recruit 
participants from the existing ASPREE-XT study. 


At which stage of the research project has this 
report been written? (Select from: 

1. Before the intervention or initiative— this 
report is prospective or describes planned 
activity 

2. Ongoing — the intervention or initiative is 
still taking place 

3. After the research project or initiative has 
occurred 


After the co-design process occurred, but before 
the proposed multi-generational research study 
has been approved or funded. 


Methods of the initiative (what is planned to be 
done, or is being reported as done) 


Participatory action research to involve elderly 
research participants in the co-design of a 
proposed multi-generational cohort study. 


Report authorship 


Name 


Jack Nunn 
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Public domain profiles, institutional pages https://scholars.latrobe.edu.au/display/j2nunn 
Open Researcher and Contributor ID (orcid.org) https://orcid.org/O000-0003-0316-3254 

Tasks in report completion Main author 

Date of report authorship 22nd July 2020 


Key contact at initiative for confirming report 
content 


Paul Lacaze, PhD, Head, Public Health Genomics 
Program, Paul.Lacaze@monash.edu 


Involvement 


Who was involved 


4. 3 academic research investigators 
5. AnASPREE participant assessor 
6. An ASPREE-XT participant 


Specific tasks of this person or group (list as many as 
possible) — including any information about why 
certain people were included or excluded in certain 
tasks 


Everyone listed above was involved in co- 
designing every stage of the process. This 
included refining wording of participant 
information, sharing views and advice about the 
process, proof-reading documents, providing 
feedback on questionnaires, analysing data, 
informing planning, presenting information to 
participants, interpreting data, and participating 
in email surveys. 


How were these people involved (what methods 
were used) 


Face to face meetings, email communication, 
shared online documents, teleconferences. 


Enablers of involvement (what do you expect will 
help these people get involved — or what helped 
them get involved) 


Giving people time to read resources. Clear 
communication about the intention of involving 
people. 


Barriers of involvement (what do you expect will 
inhibit these people from getting involved — or what 
inhibited them from getting involved). Are there any 
known equity issues which may contribute? 


Face-to-face meetings were difficult to organise. 
Some participants and participant 
representatives were elderly or lived in remote 
areas, so face-to-face meetings needed to be 
minimised where possible. 


What was the outcome or output of the involvement 
of these people? What changed as a result of 
involving people? 


Improved participant information resources, 
improved wording that is culturally appropriate, 
improved question design for interviews, 
improved learning resources for participants, 
improved co-design process. 


Which stage of the initiative were these people 
involved? (select from list of pre-defined stages or 
allow ‘other’) 


All stages 


What was the estimated financial cost for involving 
people. How much time did it take. Were there any 
costs that cannot be measured financially? 


$10,000 AUD was the estimated cost for the 
process. The total number of hours, including 
staff time was estimated to be around 200, 
including telephone interviews and excluding 
data analysis. 

Some people who attended events were unable 
to work or care for people on that day, and these 
costs were not calculated. 
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What worked well, what could have been improved? 
Was anything learned from the process of involving 
these people? 


The process took longer than expected. There 
was confusion over what ethics approval was 
required in order to involve people, especially 
people who are participants in an ongoing study. 


Involving field staff (as well as senior researchers 
and academics) provided a valuable perspective, 
as some staff knew some participants personally 
and had knowledge that senior research staff did 
not. 


Some study team members worried about over- 
burdening participants by asking them to do too 
much, however this concern did not seem to be 
backed up by the data collected, and may be 
considered a barrier to involvement. 


Mapping financial or other ‘interests’ 


Describe any financial relationship or other interest 
this person has to this project 


Three members of the study team were 
employed by Monash University during this 
process. 


Describe any conflicting or competing interests 


N/A 


Data 


Who is the data from this intervention shared with? 


It will be published open access 


How is it stored and hosted? 


It will be shared on a public domain repository. 


Who is analysing the data? 


The study team described above 


What methods will be used to analyse the data 
(including a link to any relevant code and 
information about validity) 


We used case study methodology to describe our 
experience involving participants in the co- 
design of the proposed study. We collected and 
analysed both qualitative and quantitative data 
during the involvement activities. 


We analysed data from audio recordings of 
interviews and events, meeting notes, emails, 
reflexive diary entries and survey responses of 
study investigators. Coding and thematic analysis 
of qualitative data was carried out by two 
authors independently and checked by other 
authors. 


How is information about this data disseminated? 


4. It will be published in an open access 
journal 

5. It willbe shared as an item in a 
newsletter to participants of the ASPREE- 
XT study 

6. Learning from this process will be 
presented at conferences, shared on 
social media and through other channels 
(such as podcasts). 
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Who ‘owns’ the data or claims any kind of 
‘intellectual property’ (include relevant licensing 
information) 


Monash University 


Who controls access to the data 


Monash University 


How is/will the data be ‘Findable, Accessible, 
Interoperable, Reusable’ according to the FAIR 
criteria? 


Data will be shared in the public domain and 
licensed under a Creative Commons license. 


Impacts and outcomes 

What new knowledge has been generated? (if 
appropriate, include effect size, relevant statistics 
and level or evidence) 


6. Involving participants in co-designing a 
proposed study resulted in changes to 
the design of the proposed study 

7. The process of involving people can be 
viewed as a learning experience for both 
the participants involved and study team 
members. The process changed 
participant and study team members’ 
views about the value of involvement, 
which can be viewed as an impact of 
‘transformative learning’. 


Describe how the learning or knowledge generated 
from this initiative has or will be used 


3. Knowledge from this process will inform 
the design of a future multi-generational 
study 

4. Learning from this process can inform 
future involvement activities 


How has or how will this be measured? 


Future STARDIT reports 


Who is involved in measuring this? 


The study team 


This document contains additional data relevant to the case study ‘Involving elderly research participants in the co-design of a future multi-generational cohort study’. Contact Jack.Nunn@Latrobe.edu.au - orcid.org/0000-0003-0316-3254 
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Chapter 6 - Additional files: Involving people affected by a rare 
genetic disorder in shaping future research 


Additional File 1: Data and Analysis 


About this document 


This document contains additional data relevant to the case study ‘Involving people affected by a 
rare genetic disorder in shaping future research’. In addition it contains a more detailed description 
of the data sources in this case study. This document includes the preferences mapping data 

(STARDIT-PM), and other data about this initiative’**. The corresponding Standardised Data on 
Initiatives Alpha Version (STARDIT) of the report can be found in ‘Additional File 2 - STARDIT report’. 


Survey questions 


Pre-discussion survey questions 


After participants had read the Participant Information and given consent to participate, they were 
asked to complete the following information. The questions below are worded exactly as 
participants read them. 


1. 
2. 


8. 


Full Name (if you would prefer to not use your real name you may use a pseudonym) 
Email address (Please note this needs to be a working email address. If you do not have one 
leave this blank and we will contact you by your preferred method of communication.) 
Phone number (optional) 
Any other preferred method of communication? (Please share any other preferred method of 
communication if email or phone are not preferred) 
Age 
Gender (Choose from ‘Male’, ‘Female’, ‘Transgender’, ‘Intersex’, ‘Other’, ‘Prefer not to say’) 
Educational background (tick all that apply) 

a. Middle school qualifications (up to age 16) (‘lower’) 

b. High school qualifications (ages 16-19) (‘middle’) 

c. Degree (bachelors), diploma or post-graduate (‘higher’) 

d. Ihave qualifications or professional experience in genomics (professional) 

e. Prefer not to say 
How would you describe yourself? (Please tick all that apply) 

a. A person with EGID 

b. A parent of a person with EGID 

c. A carer of a person with EGID 

d. A partner, family member or loved one of a person with EGID 

e. Other (please describe) 


f. Prefer not to say 


This document contains additional data relevant to the case study ‘Involving people affected by a rare condition in shaping future genomic research’. Contact Jack. Nunn@Latrobe.edu.au - orcid.org/0000-0003-0316-3254 
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60 
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62 
63 
64 
65 
66 
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9. 


10. Please tick which statement applies 


In which country do you live (or spend most time)? 


a. 


Ç; 


who is under 18 


who is 18+ 


expectations? 


| am over 18 and am representing myself 


genomic research should be done in the future? 


. What makes you say that? (why did you give that answer?) 


involved, the vertical a list of tasks. The horizontal was as follows) 


I am a parent, a carer, a partner, family member or loved one of someone with EGID 


I am a parent, a carer, a partner, family member or loved one of someone with EGID 


. What made you decide to respond to our invitation to participate in this project? 


. What do you hope to get out of participating in this discussion? Do you have any specific 


the quality and the relevance of the research. Involving people can also improve participant 


experience and increase participation. Who do you think should influence what kind of 


research? (For example, what tasks could people affected by EGID be involved in?) 
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. There are many benefits of involving people other than researchers in the co-design of research 


studies at every stage of the research cycle. Research suggests that involving people improves 


. Do you have any ideas about how the people from your previous answer could influence future 


. Which aspects of any future research genomic research should be influenced by the following 


(participants were presented with a grid of tick boxes, the horizontal axis being who should be 


Everyone Anyone who | Only people | Only people Only people 
(any member | might be who are who are with a 
of the public | indirectly directly participating | professional 
who is affected by affected by in the role in 
interested) the research | the research | research research 
a. All aspects mentioned below (leave others blank if ticking this) 
b. Finding questions to ask (identifying research topics) 
c. Deciding which questions to prioritize and fund 
d. Deciding how to try and answer the question (the research method) 
e. Attempting to answer the question (carrying out the research, including collecting 
information) 
f. Trying to understand if it is possible to answer the question (analyzing the information) 
g. Sharing the information that has been found, and any answers that may have emerged 


(dissemination and publication) 


This document contains additional data relevant to the case study ‘Involving people affected by a rare con 


dition in shaping future genomic resi 


earch’. Contact Jack.Nunn@Latrobe.edu.au - orcid.org/0000-0003-0316-3254 


68 
69 
70 
71 
72 
73 
74 
75 
76 
77 
78 
79 
80 
81 
82 
83 
84 


85 
86 
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h. Ensuring that any information or answers are able to be used to help people in practice, 
policy or future research (sometimes called research translation) 
i. Deciding if the way of asking the question and all the other stages of the research were 
appropriate (evaluating the research method and any impacts) 
j. Designing how people are involved in the research 
17. Have you ever participated in research in the past? (by participation, we mean as a research 
subject — for example part of a trial) 
a. Yes 
b. No 
c. Prefer not to say 
d. Unsure 
18. Have you ever participated in research in the past? (by participation, we mean as a research 
subject — for example part of a trial) 
a. Yes 
b. No 
c. Prefer not to say 


d. Unsure 
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Post discussion survey questions 


1. 


10. 


11. 
12. 


13. 


How would you rate the following? (chose from ‘Excellent’, ‘Somewhat good’, ‘Neither good 
nor bad’, ‘Somewhat poor’, ‘Extremely poor’) 
a. Your overall experience of participating in the online discussion 


b. Your assessment of how we conducted the survey and discussion format 


c. The support you received to be involved (for example, practical support such as 
instructions for using the online tools)? 
d. Information and learning materials you were given before the event 


Did you feel you meaningfully contributed to the discussion? 


a. Yes 
b. No 
c. Unsure 


Is there anything in particular you liked or thought was helpful about how the discussion was 
conducted? 

Is there anything you didn’t like, thought was unhelpful or could have been improved about 
how the discussion was conducted? 

Do you have any other thoughts, ideas or comments? 

Would you like to be updated about the progress of the research and offered chances to be 
involved where possible? (Chose ‘yes’ or ‘no’) 

Did you have any expectations from participating in this research that were met or not met? 
Have any of your views and perspectives about involving people in genomic research 
changed since participating in this research? If so, please describe. 

There are many benefits of involving people other than researchers in the co-design of 
research studies at every stage of the research cycle. 

Research suggests that involving people improves the quality and the relevance of the 
research. Involving people can also improve participant experience and increase 
participation. Who do you think should influence what kind of genomic research should be 
done in the future? 

What makes you say that? (why did you give that answer?) 

Do you have any ideas about how the people from your previous answer could influence 
future research? For example, what tasks could people affected by EGID be involved in? 
Which aspects of any future research genomic research should be influenced by the 
following (participants were presented with a grid of tick boxes, the horizontal axis being 


who should be involved, the vertical a list of tasks. The horizontal was as follows) 
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124 
125 
126 
127 
128 
129 
130 
131 
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134 
135 
136 
137 
138 
139 
140 
141 
142 
143 
144 
145 
146 
147 
148 
149 
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152 
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15 
16 


17. 


18 
19 
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Everyone Anyone who | Only people | Only people Only people 
(any member | might be who are who are with a 
of the public | indirectly directly participating | professional 
who is affected by affected by inthe role in 
interested) the research | the research | research research 
a. All aspects mentioned below (leave others blank if ticking this) 
b. Finding questions to ask (identifying research topics) 
c. Deciding which questions to prioritize and fund 
d. Deciding how to try and answer the question (the research method) 
e. Attempting to answer the question (carrying out the research, including collecting 
information) 
f. Trying to understand if it is possible to answer the question (analyzing the 
information) 
g. Sharing the information that has been found, and any answers that may have 
emerged (dissemination and publication) 
h. Ensuring that any information or answers are able to be used to help people in 


j. 


practice, policy or future research (sometimes called research translation) 
Deciding if the way of asking the question and all the other stages of the research 
were appropriate (evaluating the research method and any impacts) 


Designing how people are involved in the research 


. Full Name (Optional- if you would prefer to not use your real name you may use a 


pseudonym) 


. Email address (optional) 


. Phone number (optional) 


Age 


. Gender (Choose from ‘Male’, ‘Female’, ‘Transgender’, ‘Intersex’, ‘Other’, ‘Prefer not to say’) 


. Educational background (tick all that apply) 


a. 
b. 
C. 
d. 


e. 


Middle school qualifications (up to age 16) (‘lower’) 

High school qualifications (ages 16-19) (‘middle’) 

Degree (bachelors), diploma or post-graduate (‘higher’) 

| have qualifications or professional experience in genomics (professional) 


Prefer not to say 


20. How would you describe yourself? (Please tick all that apply) 


This document contains additional data relevant to the case study ‘Involving people affected by a rare condition in shaping future genomic res 


a. 


b. 


A person with EGID 


A parent of a person with EGID 


earch’. Contact Jack.Nunn@Latrobe.edu.au - orcid.org/0000-0003-0316-3254 


153 
154 
155 
156 
157 


158 
159 


c. A carer of a person with EGID 

d. A partner, family member or loved one of a person with EGID 
e. Other (please describe) 

f. Prefer not to say 


21. In which country do you live (or spend most time)? 


This document contains additional data relevant to the case study ‘Involving people affected by a rare condition in shaping future genomic research’. Contact Jack. Nunn@Latrobe.edu.au - orcid.org/0000-0003-0316-3254 
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160 
161 
162 
163 
164 
165 
166 
167 
168 
169 
170 
171 
172 
173 
174 
175 
176 
177 
178 
179 
180 
181 
182 
183 


Facilitator survey questions 


1. 
2. 


oO oN DD 


10. 


11. 


12. 


13. 


Please describe your tasks in the process of involving people in planning of the study 
What did you learn from the process of involving participants in the research planning 
phase? 

Please describe specifically what worked well or was useful about the way the study was 
conducted (including how people were involved) 

Please describe specifically what did not work well or was not useful about the way the 
study was conducted (including how people were involved) 

Were there any barriers or enablers to conducting the study or involvement activities? 
(institutional or otherwise) 

Do you think the involvement activity achieved its intended aim(s)? 

Do you think the study achieved its intended aim(s)? 

Do you have any advice to other researchers planning involvement for their research? 
Do you have any advice to other researchers planning to involve people using online 
discussions? 

Describe the impact you think involving people had (positive/negative - on the research, 


staff or participants) 
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Who do you think should influence the kind of human genomic research done in the future, 


and why? (e.g. the public, participants of research studies, doctors, school children, 
politicians etc) 

Which stages of future genomic research should be influenced by people other than 
researchers (if any)? (e.g. concept planning of new studies, study design, conducting the 
research, presenting the results etc) 


Other comments 
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STARDIT Preference Mapping (STARDIT-PM) 


This table uses the Alpha version of the Standardised Data on Initiatives Preference Mapping 
(STARDIT-PM) to categorise the data into certain areas 1%. Preferences were recorded from all data 
sources, including the initial survey, online discussion with participants, online Facilitator 
discussions, follow-up surveys with participants and with facilitators. Facilitator comments were only 
included from one Facilitator with personal experience (KG), and comments from the other 

Facilitator were excluded from analysis (JN). If the same participant made the same point at 

different stages, this was counted as one view. The standardised categorisation is intended to 
facilitate comparison with other studies. Accordingly, the content may be similar to other sections of 
the qualitative thematic analysis. 


STARDIT-PM area 
and quantitative 
data 

Views on who should 
be involved: 


17 participants 
shared views about 
who should be 
involved 


Views on specific 
tasks people 
involved could do: 


10 participants 
shared views about 
specific tasks people 
involved could do 


Views on modes of 
communication: 


2 participants 
shared views about 


This document contains additional data relevant to the case study ‘Involving people affected by a rare condition in shaping future genomic res 


Qualitative summary 


One participant wrote that asking ‘Who should be ‘excluded’ is a helpful 
starting point when answering this question’ [P3]. Participants 
contributed 17 different statements saying patients and their families 
should be involved. The word ‘collaboration’ was used to describe how 
people should work together. Besides patients and families and those 
‘directly affected’ [P3], other ‘specialists’ and groups were mentioned 
by participants [P23]. These included including doctors, medical 
professionals, researchers, patient advocacy groups, immunologists, 
gastroenterologists and IT experts. One participant stated ‘drug 
companies will have a part to play’ and went on to say ‘this would need 
supervision and strict guidelines’ [P9]. 


Participants shared multiple tasks which they felt patients should be 
involved in. Identifying topics and ‘what’s a priority’ was mentioned 5 
times [P3]. One participant wrote ‘medical researchers should take the 
opportunity to be guided more from the patient themselves on an 
idea/direction for a research project rather than the other way round’ 
[P21]. Involving people affected in setting outcomes was also 
mentioned by participants, including being involved in saying what 
‘would be useful’ to patients [P22]. Involving people affected ‘at the 
design stage’ of research was mentioned, with two participants 
mentioning surveys as an example. For example, a ‘quality of life 
survey’ was mentioned as a way people could make sure the right 
questions were being asked [P21] 

[P25]. Fundraising, campaigning and ‘advocating’ were mentioned as 
ways of ‘raising awareness’ by two participants [P5] [P9]. Being involved 
in ‘doing’ research was mentioned by one participant [P3]. 

Being involved in working with insurance companies and government to 
explore the legal, financial and privacy impacts of how a diagnosis 
might affect people was mentioned by one participant [P3]. One 
participant mentioned involving patients in discussions about health 
technology assessment, including ‘health economics and the 
consequences’ [P3]. 

Two participants mentioned websites, with ‘public government 
research websites’ and ‘patient advocacy groups’ as ways of advertising 
opportunities for involvement or participation. Face to face research 
and online surveys were also mentioned as other modes. Online 
discussion, face to face communication were also mentioned. 


earch’. Contact Jack.Nunn@Latrobe.edu.au - orcid.org/0000-0003-0316-3254 
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preferred 
communication 
modes 


Views on what 
methods should be 
used to involve 
people: 


6 participants 
shared views about 
what methods 
should be used to 
involve people 


Views on facilitators 
of involvement: 


10 participants and 
3 facilitators shared 
views about 
facilitators’ of 
involvement 
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One participant mentioned surveys, stating that ‘short surveys’ 
completed on computers or phones are ‘easy’ and cost effective, 
especially while people ‘wait for their medical appointment’ [P25]. 
Groups such as ‘volunteer committees’ were suggested as a method. 
One participant suggested that public awareness ‘facilitates 
involvement’ [P25] and that people sharing stories can be a source of 
‘support or inspiration’, in particular ‘public figures’ [P25]. Online and 
face to face support groups were identified as an area for ‘uncovering 
trends’, the first stage in identifying research topics [P3] 

‘Community discussions’ [P16] and ‘working collaboratively’ with 
professors and ‘those directly affected’ [P3] was mentioned by two 
participants. Similarly one participant stated ‘patient advocacy groups 
partnering with researchers is valuable for all involved’ [P28]. One 
participant stated she liked ‘focus groups as face to face’ as when 
working in an online group the ‘intent of the writer’ can be interpreted 
differently to as it was intended (for example, presuming an incorrect 
tone of voice)’ [P21]. Another participant preferred a combination of 
face to face events once or twice a year with most work being done by 
teleconferencing, owing to geographical separation [P3]. Online video 
teleconferencing was also suggested as a good method as participants 
‘get a much better feel for people and their thoughts because you can 
see them (get all the cues), then break away for specific follow-up’ [P3]. 
Another participant agreed a ‘two stage’ approach would be good, 
starting with involving people online, then meeting face to face [P25]. 
Ensuring online discussions have a mixture of both open and closed 
questions allows the open questions to ‘create another idea’ and novel 
discussion [P3]. 


One participant noted that ‘Participants need to be able to contribute 
without putting their personal situation at risk’. One participated noted 
that ‘more respect for the patient and their family, their knowledge & 
experience’ would facilitate involvement. Two participants noted 
‘Power and knowledge’ as facilitators [P20], highlighting not having 
‘payment for access’ to information as a facilitator [P3]. Emotional 
connection to an issue can be both a facilitator and a barrier to 
involvement, with a ‘balance’ highlighted as a facilitator of involvement 
[P21]. For those who are unable to travel or live in remote areas, online 
discussions are ‘good to help’ people get involved, with one participant 
stating ‘It is hard for us to help with research’ if it is not in their area 
and they are ‘unable to afford to travel’ [P20]. Good facilitation of face 
to face events was highlighted by one participant as face to face 
discussions ‘can be endless so need to be governed well’ [P3]. Public 
awareness facilitates involvement, especially stories from people 
affected. This also can be a source of ‘support or inspiration’ for some 
people [P25]. 
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Views on barriers of 
involvement: 


5 participants and 3 
facilitators shared 
views about barriers 
of involvement 


Six participants reported specific things about the way this study was 
conducted that facilitated their involvement. One participant ‘enjoyed 
the interaction’ and four found it ‘helpful’ to have ‘links with 
information about genomics’ relevant to the topic threads’ to ‘explain 
a concept’ [P21, P3, P29, P28]. One participant also stated ‘being able 
to read others thought processes on each topic’ helped [P3]. Another 
responded that responses in the discussion ‘provoked further 
questions that made me think in new directions’ [P25]. Another 
participant added that other participants were ‘supportive, positive 
and open’ [P3]. One participant noted that the flexibility of being able 
to join at any time ‘was great’ as she was often ‘busy caring for her son 
with EoE and travelling’. They stated that the way the discussion was 
hosted meant that they ‘could still catch up and learn things’ and also 


410 


provide input [P30]. Two participants mentioned having different topics 


threads with a lead question was helpful and a ‘good format’ [P29] 
[P28]. One participant stated ‘This process has been really interesting 
because we can only write so much, get a variety of input which makes 
us think’ [P3]. 


Facilitators reported that they ‘needed more support and advice than 
expected’, stating that the training and ongoing support given before 
and during facilitation was essential. One Facilitator reported that 
‘touching base’ and learning the experiences of other facilitators and 
feeling ‘part of the team’ all facilitated their facilitation of online 
discussion. 

One participant stated barriers included ‘payment for access’ to 
information, such as paywalls for peer-reviewed information [P3]. 
Researchers promising ‘feedback that never came’ was considered a 
barrier to involvement by one participant [P25]. One participant stated 
that barriers included representatives being ‘undervalued and under- 
supported.’ [P3]. One participant noted that issues which have been 
identified should be addressed by ‘the Ministry’ and as a result of 
inaction, volunteers were ‘working on extra time’. [P3]. Online 
discussions ‘can be viewed differently to the intent of the writer’ 
[P21]. One participant highlighted that privacy was an issue when 
involving people in research using focus groups online or face-to-face. 
They stated ‘some people might share more if they didn't have a 
broader audience - especially people they don't know’. [P21]. Two 
participants reported specific things about the way this study was 
conducted that were barriers to their involvement. One participant 
reported that although they like the format of the discussion, they 
‘didn't like the platform it was conducted on’ as it was not ‘user 
friendly’ [P15]. Another reported that the pace of new questions being 
added (around one every two days) was ‘too quick’ and done ‘before a 
number of people had a chance to answer’. One participant felt 
facilitators sometimes did not accurately summarise participants’ 
comments, and that some participants might feel reluctant to correct 
a Facilitator [P3]. 


Facilitators reported finding it a challenge to separate ‘personal’ 
experiences when facilitating. One Facilitator noted ‘Separating out my 
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parent role and Facilitator role has been hard. Harder than | thought it 
would be’. One Facilitator also noted ‘The discussion was too rapid. The 
group needed more time to work through the process of the online 
conversation. It felt too rushed.’ 


Views on which One participant noted that involving people in the design stage is more 
stage of the research practical as once research begins ‘you can’t change direction or 

people should be question’ [P3]. Two participants stated that identifying topics and 
involved: research development were appropriate stages for people to be 

3 participants views involved. One participant noted that people ‘must influence the 

on what the research agenda including the questions, how they are funded, 


outcome or output research design, data analysis, interpretation and dissemination of 
of the involvement results’ [P16] 
could be 


Who should the data from this project be shared with?: 


0 participants shared views about who data from this project shared with 


Views on what the outcome or output of the involvement could be: 


0 participants shared views about what the outcome or output of the involvement could be 


Number of participants who shared views on STARDIT-PM 
categories 


Who should the data from this project shared with? 


Views on what the outcome or output of the 
involvement could be 


Views on what the outcome or output of the 
involvement could be 


Views on barriers of involvement 

Views on facilitators of involvement 
Views on what methods should be used 
Views on modes of communication 


Views on who should be involved 


| 
E) 
SSS aa 
ae) 
Views on specific tasks of this person or group BE 
SSS 


oO 
N 
A 
fon) 
(oe) 
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oO 
= 
N 
m 
P 
= 
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= 
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Data 


This table summarises all the data sources used for the case study. 
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Data Category 
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Data point description 


Diary 


ausEE research diary of lead investigator (JN) — including 
reflections 


Emails and meeting notes 


Email, meeting notes and Notes from planning and 
discussion. This included involvement of participant 
representatives in co-refining study design. 


Online pre-discussion survey 


Informed consent and pre-discussion survey data 


Learning resources for participants 
and facilitators 


Learning resources giving information about genomics and 
using Loomio 


Online discussion with participants 


Text data from online Loomio discussion with participants 


Online discussion with facilitators 


Text data from online Loomio discussion between facilitators 
of two parallel studies 


Online post-discussion survey 


Post-discussion survey data from participants 


Follow up survey for facilitators 


Post-discussion survey data from facilitators 
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Demographic information 
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Category Pre-discussion survey Post discussion Survey 
Gender 

Female 28 8 
Male 1 0 
Age 

20-24 years 1 0 
25-29 years 1 0 
30-34 years 3 1 
35-39 years 12 0 
40-44 years 7 2 
45-49 years 3 1 
50-54 years 3 1 
Educational background (‘highest' 

only counted) 

Prefer not to say 1 0 
| have qualifications or professional 

experience in genomics 1 0 
Degree (bachelors), diploma or 

post-graduate (‘higher’) 26 6 
High school qualifications (ages 16- 

19) 2 1 
Middle school qualifications (up to 

age 16) (‘lower’) 1 0 
How would you describe yourself? 

A carer of a person with EGID 1 

A parent of a person with EGID 22 

A partner, family member or loved 

one of a person with EGID 1 0 
A person with EGID 5 1 
A person with EGID and a parent of 

a person with EGID 1 0 
In which country do you live (or 

spend most time)? 

Australia 28 6 
New Zealand 1 0 
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Widening and narrowing for each question asked in pre and post survey 
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Who should influence which aspects of research? | Change to | No change | Change to 
wider narrower 

Finding questions to ask 3 1 1 

Deciding which questions to prioritize and fund 3 1 1 

Deciding how to try and answer the question (the | 3 2 0 

research method) 

Attempting to answer the question (carrying out 2 2 1 

the research, including collecting information) 

Trying to understand if it is possible to the answer | 2 3 0 

the question (analysing the information) 

Sharing the information that has been found, and | 3 2 0 

any answers that may have emerged 

(dissemination and publication) 

Ensuring that any information or answers are able | 4 1 0 

to be used to help people in practice, policy or 

future research (sometimes called research 

translation) 

Deciding if the way of asking the question and all 2 1 0 

the other stages of the research were appropriate 

(evaluating the research method and any impacts) 

Designing how people are involved in the research | 2 1 0 


Change totals 
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Participant experience 


ie) 


Excellent 
Somewhat good 


a 


Excellent 11 
Somewhat good 3 
Neither good nor bad 1 


Excellent 11 
Somewhat good 4 


Excellent 11 
Somewhat good 5 
Neither good nor bad 1 
Yes 8 
Unsure 7 


Learning resources 


A number of different learning resources were shared with participants at different stages of the 
process. This included a short 60 second online video about the study, giving information about the 
context and purpose*’”, a one page infographic summary of a scoping review about genomics 
research®®®, and a short two-page summary of genomics and contemporary research relating to EoE 
was co-created with ausEE, the study team and experts in genomics?®”. 

Learning resources were both co-created and selected by the investigator team, working in 
partnership with the Australian Genomics Health Alliance and co-refining the selection with 
potential participants. In addition, in order to support the Enablers in providing good quality 
information, a number of were curated into a list to be available for Enablers to share during the 
online discussion, if they became relevant to aspects of the discussion in order to help inform 
people. The table below provides a summary of which learning resources were shared at which stage 
of the process. 
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English 


Stage Title Media Summary Authorship 
Stage 2: Before What is genomic Portable Document | Simple infographic explaining the Australian Genomics 
consent testing? Format (PDF) basics of genomics research in plain Health Alliance>® 


What is genomics? 


Online video 
animation with 
audio narration 
and subtitles 


A 6 minute video outlining the 
principles of genomics research 


Genome BC™42 


Stage 3: Before online 
discussion 


Definitions and 
explanations 


PDF 


These definitions and explanations 
were used as a glossary to explain the 
main concepts of this research project. 


Investigator team 


Genomics and 
involvement 


Online video with 
hard-coded text 


A 2 minute video exploring why people 
should be involved in genomics 


Jack Nunn?87 


Infographic summary 
of scoping review 


PDF (infographic — 
images and text) 


A one page summary of the main 
findings from a recent scoping review 
about involving people in genomics? 


Jack Nunn et al?88 


A summary of EGID and 
relevant genomics 
research 


PDF (text with 
hyperlinks) 


A co-created learning resource 
updating people on what is known so 
far, what research is currently being 
done and what might be the future 


Jack Nunn et al?89 


Guide to using Loomio 


PDF (text with 


A co-created learning resource giving 


Investigator team 


available to facilitator 


conditions 
(chapter) 


wanted more information about 
inherited conditions 


hyperlinks) practical advice for using the online 
discussion platform Loomio 
Additional resources | Inheriting genomic Webpage An additional resource if participants U.S National Library of 


Medicine>® 


Data in the 100,000 
Genomes Project 


Online video 
animation with 


An example of the bioinformatic 
pathway (specific to Genomics 


Genomics England>°* 
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voice over and 


England) but generalisable (talks about 


subtitles access review committee) 
Ethical issues in human | PDF (text with Additional resource for a relevant Centre for Genomics 
genomics and hyperlinks) ethics discussion from a medical Education®© 


genomics 


perspective 


Genes, DNA and cancer 


Webpage (text 
with hyperlinks) 


Good plain English information about 
genes in relation to cancer. 


Cancer Research UK>®6 


How to Share Genomic 
Test Results With 
Family 


Webpage (text 
with hyperlinks) 


Good information about sharing 
genomic test results with family. 


American Society of 
Clinical Oncology*®” 


How do you sequence 
a human genome? 


Image file 
(infographic — 
images and text) 


Infographic about the stages of 
genome sequencing 


Genomics England>® 
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Additional File 2: Standardised Data on Initiatives (STARDIT) report — Alpha 
Version: Involving people affected by a rare genetic disorder in shaping 


future research 


About this report 


This report uses the Standardised Data on Initiatives Alpha version (STARDIT)?%. An Alpha version of 
this STARDIT report is also hosted online in machine readable format.° 


A machine readable structured data version of this report can be found here: 


https://www.wikidata.org/wiki/Q100403236 
STARDIT Report Alpha Version 


Identifying information 


Initiative name 


Involving people affected by a rare genetic disorder in 
shaping future research 


Geographic location or scope 


Australia 


Date range (planned start and end dates of 
initiative) 


2017-2020 


Purpose of the initiative 


Participatory action research to involve people 
affected by a rare disease in shaping future research, 
by using online discussions. 


Organisations or other initiatives involved (list 
all if multi-centre) 


3. School of Psychology and Public Health, La 
Trobe University 


Funding sources 


School of Psychology and Public Health, La Trobe 
University 


Clinical trial registration details (if applicable) 


N/A 


Ethics approval (if applicable) 


The La Trobe University Human Ethics Committee 
approved this study. 

Project number: HEC18242 

Project Title: Genomics Research and Involving 
People: ausEE 


Other relevant information (free text) 


This report describes involving people affected bya 
rare disease in shaping future research, by using 
online discussions exploring how they would like to 
be involved in future research. 


At which stage of the research project has this 
report been written? 


After the participatory action research occurred. 


Methods of the initiative (what is planned to be 
done, or is being reported as done) 


The research process was co-designed using a 
participatory action research method to involve 
people affected by a rare disease in the co-design of 
online discussions to explore future genomic research 
with members of the group. 


Report authorship 


Name 


Jack Nunn 


Public domain profiles, institutional pages 


https://scholars.latrobe.edu.au/display/j2nunn 


Open Researcher and Contributor ID (orcid.org) 


https://orcid.org/O000-0003-0316-3254 
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Tasks in report completion 


Main author 


Date of report authorship 


24* July 2020 


Key contact at initiative for confirming report 
content (include institutional email address) 


Jack Nunn, PhD researcher, School of Psychology and 
Public Health, La Trobe University, 


jack.nunn@latrobe.edu 


Involvement 


Who was involved or how would you label 
groupings of those involved 


Group 1: Academic research investigators (Jack Nunn 
and Paul Lacaze) 

Group 2: People affected by the rare disease 
representing the charity ausEE with experience of 
academic research (Kylie Gwynne) 

Group 3: People affected by the rare disease 
representing the charity ausEE (Sarah Gray) 

Group 4: People affected by the rare disease who are 
members of the online community and participated 
in the study 


How many people were in each grouping label? 


Group 1: 2 
Group 2:1 
Group 3:1 
Group 4: 25 


Specific tasks of this person or group (list as 
many as possible) — including any information 
about why certain people were included or 
excluded in certain tasks 


Group 1 and 2: Involved in co-designing every stage 
of the process, analysing data and member checking 
during the thematic analysis 

Group 3: Involved in co-designing the recruitment 
and giving feedback on the proposed study design 
and as an author of the paper. 

Group 4: Invited to give feedback on the paper 


How were these people involved (what 
methods were used) 


Group 1 - 3: Face to face meetings, video calls, email 
communication, shared online documents, 
teleconferences. 

Group 4: Invited to give feedback on paper and be 
acknowledged for contribution 


Enablers of involvement (what do you expect 
will help these people get involved — or what 
helped them get involved) 


Giving people time to read resources. Clear 
communication about the intention of involving 
people. Have multiple modes of communication for 
involving people. 


Barriers of involvement (what do you expect 
will inhibit these people from getting involved — 
or what inhibited them from getting involved). 
Are there any known equity issues which may 
contribute? 


Face-to-face meetings were difficult to organise. The 
study team were located in different states of 
Australia. Unclear communication about intentions 
and purpose of the involvement contributed to 
confusion (explaining how involvement is distinct 
from participation was challenging). Ensuring those 
involved had enough time to give feedback was also a 
challenge. 
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What was the outcome or output of the 
involvement of these people? What changed as 
a result of involving people? 


Improved participant information resources, 
improved wording that was culturally appropriate 
(using terminology preferred by the group to describe 
themselves), improved online discussion, improved 
learning resources for participants, improved co- 
design process. 


Which stage of the initiative were these people 
involved? (select from list of pre-defined stages 
or allow ‘other’) 


Group 1 and 2: All stages 
Group 3: Co-design, evaluation, dissemination 
Group 4: evaluation, dissemination 


What was the estimated financial cost for 
involving people. How much time did it take. 
Were there any costs that cannot be measured 
financially? 


SO AUD — people volunteered their time. 
The total number of hours volunteered (excluding 
participation) is estimated to be 75. 


What worked well, what could have been 
improved? Was anything learned from the 
process of involving these people? 


The co-design process took longer than expected 
owing to ethical ‘grey areas’ with no clear instruction 
on whether ethics approval was required to involve 
people in co-design. As a result an ethics application 
was made and subsequent feedback from the co- 
design process was integrated using modifications to 
the ethics application. 


Involving potential participants in co-defining 
language used to describe the group of people 
affected helped ensure that language was acceptable 
and appropriate. 


Mapping financial or other ‘interests’ 


Describe any financial relationship or other 
interest this person has to this project 


Two investigators are affected by the rare disease, 
which is why they were invited to be part of the 
study 


Describe any conflicting or competing interests 


N/A 


Data 


Who is the data from this intervention shared 
with? 


It will be published open access in peer reviewed 
journals with identifying information removed in 
order to anonymise it as much as possible. 


How is it stored and hosted? 


It will be shared on a public domain repository. 


Who is analysing the data? 


Group 1-3: The study team described above 
Group 4: participants were invited to review the 
analysis and give feedback to ensure they felt it 
reflected their experience of the process 


What methods will be used to analyse the data 
(including a link to any relevant code and 
information about validity) 


We used case study methodology to describe our 
experience involving participants in the co-design of 
the proposed study. We collected and analysed both 
qualitative and quantitative data during the 
involvement activities. 


We analysed data from online surveys and online 

discussions with participants. In addition, data from 
the study team communications was included, such 
as meeting notes, emails, reflexive diary entries and 
survey responses of study investigators. Coding and 


420 


421 


thematic analysis of qualitative data was carried out 
by two authors independently and checked by other 
authors. 


How is information about this data 
disseminated? 


7. It will be published in an open access journal 

8. It will be shared with participants of the 
research and also other members of the 
sibling group who have joined it since the 
study commenced 

9. Learning from this process will be presented 
at conferences, shared on social media and 
through other channels (such as podcasts). 


Who ‘owns’ the data or claims any kind of 
‘intellectual property’ (include relevant 
licensing information) 


Confidential data collected as part of the study is 
stored according to laws and the data access plan 
approved by La Trobe University. 


The authors maintain ‘ownership’ of the data in the 
paper and is shared under the Creative Commons 
license used by the publishing journal. 


Who controls access to the data 


The study team, La Trobe University and participants 
will be involved in any future data access decisions. 


How is/will the data be ‘Findable, Accessible, 
Interoperable, Reusable’ according to the FAIR 
criteria? 


Data will be shared in the public domain and licensed 
under a Creative Commons license. 


Impacts and outcomes 


What new knowledge has been generated? (if 
appropriate, include effect size, relevant 
statistics and level or evidence) 


1. Involving participants in co-designing the 
research process resulted in a number of 
changes to the study design, including 
improving language used in recruitment and 
learning resources 

2. The process of involving people can be 
viewed as a learning experience for both the 
participants involved and study team 
members. The process changed participants’ 
views about who should be involved, which 
can be viewed as an impact of 
‘transformative learning’. 


What was learned 


Involving people in online discussions about 
involvement in research changes people’s views 
about who should be involved in research, including 
participants ‘widening’ their views about who should 
be involved in research to include more people. 


Knowledge translation 


1. Knowledge from this process will inform the 
design of a future genomic research 

2. Learning from this process can inform future 
involvement activities 


Outcomes 


Learning from this process informed subsequent 
discussions in the charity ausEE about involvement in 
research, including proposed improved ways of 
involving people 
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How has or how will this be measured? Future STARDIT reports 


Who is involved in measuring this? The study team and participants 
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Additional File 3: GRIPP2 report for ‘Involving people affected by a rare condition in shaping future genomic research’ 


This report has been completed using the ‘GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research’ available 
https://doi.org/10.1136/bmj.j3453 


GRIPP2 short form 


Report the aim of PPI Participatory action research to involve people affected by a rare disease in shaping future research, by using 


1: Aim 
in the study online discussions. 


Provide a clear The research process was co-designed using a participatory action research method to involve people affected by a 
description of the rare disease in the co-design of online discussions to explore future genomic research with members of the group. 
methods used for PPI Participants were also involved in analysing the data and checking the final version of the paper. 

in the study 


2: Methods 


The input of the representatives during the planning and co-design stage had clear positive impacts, particularly in 
improving educational resources and ensuring the online discussion was advertised using wording appropriate to 
the existing online community. For example, representatives from ausEE helped change the study design to include 


explicit opportunities for participants to learn more about genomics and EoE, avoiding participation being 
perceived as having a one-way benefit. During the co-design process it was also decided to exclude people who 
Outcomes—Report the | were under 18 and people who stated they were representing someone who was over 18, as people who were 18 
results of PPI in the and over had the choice to represent themselves. 
3: Study results study, including both 
positive and negative Enablers of involvement: Giving people time to read resources. Clear communication about the intention of 
outcomes involving people. Have multiple modes of communication for involving people. 


Barriers of involvement: Face-to-face meetings were difficult to organise. The study team were located in different 
states of Australia. Unclear communication about intentions and purpose of the involvement contributed to 
confusion (explaining how involvement is distinct from participation was challenging). Ensuring those involved had 
enough time to give feedback was also a challenge. 


Section and topic 


4: Discussion and 
conclusions 


5: 
Reflections/critical 
perspective 
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Outcomes—Comment 
on the extent to which 
PPI influenced the 
study overall. Describe 
positive and negative 
effects 


Comment critically on 
the study, reflecting on 
the things that went 
well and those that did 
not, so others can 
learn from this 
experience 


Involvement improved participant information resources, improved wording that was culturally appropriate (using 
terminology preferred by the group to describe themselves), improved online discussion, improved learning 
resources for participants, improved co-design process. 


Involving potential participants in co-defining language used to describe the group of people affected helped 
ensure that language was acceptable and appropriate. 


Involving participants in co-designing the research process resulted in a number of changes to the study design, 
including improving language used in recruitment and learning resources. 


The process of involving people can be viewed as a learning experience for both the participants involved and study 
team members. The process changed participants’ views about who should be involved, which can be viewed as an 
impact of ‘transformative learning’. 


The co-design process took longer than expected owing to ethical ‘grey areas’ with no clear instruction on whether 
ethics approval was required to involve people in co-design. As a result an ethics application was made and 
subsequent feedback from the co-design process was integrated using modifications to the ethics application. 


Involving potential participants in co-defining language used to describe the group of people affected helped 
ensure that language was acceptable and appropriate. 


Involving participants in co-designing the research process resulted in a number of changes to the study design, 
including improving language used in recruitment and learning resources. 


The process of involving people can be viewed as a learning experience for both the participants involved and study 
team members. The process changed participants’ views about who should be involved, which can be viewed as an 
impact of ‘transformative learning’. 


Involving people in online discussions about involvement in research changes people’s views about who should be 
involved in research, including participants ‘widening’ their views about who should be involved in research to 
include more people. 


Chapter 7 - Additional files: Co-designing genomics research with a 
large group of donor-conceived siblings 


Additional File 1: Data and Analysis 


About this document 


This document contains additional data relevant to the case study ‘Co-designing genomics research 
with donor-conceived siblings’*. In addition it contains a more detailed description of the data 
sources in this case study. This document includes the preferences mapping data (STARDIT-PM), and 
other data about this initiative’®*. The corresponding Standardised Data on Initiatives Alpha Version 
(STARDIT) of the report can be found in ‘Additional File 2 - STARDIT report’. The ‘living’ STARDIT Beta 
version’? report which relates to this project can be found in the references7?”. 


Definitions of terms 


We have used consistent language to describe concepts throughout this case study. The list below 
defines important terms used throughout. 


Involvement — The words ‘involvement’ or ‘being involved’ describe the concept of people being 
‘involved’ in research. This is when research is carried out ‘with’ people rather than ‘on’ them.2°° 
‘Involvement’ can also be defined as when other people aside from the research team, such as the 
public, patients, research participants and other stakeholders, actively contribute to the research 
process.?” It is the ‘active involvement’ in shaping and guiding research, rather than only 
providing data.8%322323 


Engaged — participants in the online discussions are described as ‘engaged’ if they are reading and 
commenting in discussions, and ‘disengaged’ if they are not. 


Enablers — enablers are things which ‘facilitate’ certain things happening. For example, something 
which enables someone to participate in research. 


Facilitator —in this article it refers to a person facilitating online discussions. For example, 
“facilitators shared views about enablers of involvement” 


Online community — as the half-siblings discovered one another through various direct-to- 
consumer ancestry services, they self-created an online community which used a mailing list to an 
email group as the mode of communication. Participants were recruited from this online 
community. 


Online discussion — the study team recruited participants to an online discussion, specifically 
created for this study and hosted on the secure platform Loomio. 


Participant — a person who participated in the process of sharing views and perspectives about 
the genomics research, including sharing views about preferences for any future involvement. 


Participatory action research (PAR) - is an umbrella term which describes a number of 
related approaches, including forms of action research which embrace a participatory 
philosophy. Concepts such as ‘co-design’, ‘co-creation’ and ‘co-production’ describe 
involving people in the respective tasks of designing a project or creating a learning 
resource, and can be considered as part of participatory action research™. It is a process 
where researchers, relevant stakeholders and sometimes the public “work together, 
sharing power and responsibility from the start to the end of the project”®°, including 
knowledge generation and translation®. 


Potential participant — before inviting people to become participants, it was necessary to involve 
a number of potential participants to help advise and plan the process. 


Shared Ancestry groups -refers to people who have shared ancestors and have been grouped 
into these sub-populations by genomic researchers. 


Sibling group - refers specifically to the community of shared interest defined by people who 
shared the same sperm-donor father, Bertold Wiesner. 
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Stakeholder — this term includes anyone who has a ‘stake’ in the research, in particular those who 
have important knowledge, views or perspectives that should be taken into account.2°° In this 
paper it refers to participants, representatives, patients, parents and carers of patients, potential 
patients and the study team (including researchers and representatives) and the wider public. 
The study — the study refers to the formal research described in this case study, which was 
overseen by the ‘Ethics, Integrity and Biosafety team’ team at La Trobe University and the La 
Trobe University Human Research Ethics Committee. 

Study team — this process was guided by the study team, who consisted of academic researchers 
and members of the sibling group. The work of the study team was also advised and overseen by 
the ‘Ethics, Integrity and Biosafety team’ team at La Trobe University and the La Trobe University 
Human Research Ethics Committee. 

Sub-populations - This term refers to any grouping of people below the population level. 
Groupings can include communities of shared interest defined by shared genetic variation. These 
can include groupings of people who are half-siblings. 

The process — this term will be used to describe both the study and the co-design process which 
involved members of the sibling group by inviting them to share views and perspectives about 
genomics research, including sharing views about preferences for any future involvement. 


Case study background and context 


The practice of artificial insemination existed in a legal and ethical grey area for many years*”’. For 
example, in the UK in the 1950s, the legitimacy of children conceived from a donor father was 
unclear as husbands were registered as the fathers, which was legally an offence’. Despite the 
recognised ‘immense social, moral and medical questions’ raised by this process, there was no 
legislative oversight and the practice was self-regulated by the individuals managing clinics carrying 
out the procedure. Members of the UK’s House of Lords suggested that artificial insemination of 
married women with the husband’s consent be classified as adultery as late as 1954*©°. Such 
contemporary attitudes created potential ethical, legal and social issues for families conceiving in 
this way. Subsequently, there was a requirement for discretion for all involved, including the 
identities of donors which were surrounded by ‘complete secrecy’*”. Additionally, it was the view of 
some doctors that parents should not know the identity of the donors as it was ‘incompatible with 
secrecy’, Accordingly, many parents were encouraged to never disclose the paternity to the 
offspring. 

The total number of donor conceived people in the UK by 1958 was estimated to be 7500, and 
100,000 in the United States. One pioneering clinic mentioned in debates of the UK Parliament in 
the 1950s was the Dr Mary Barton’s medical practice, which operated in London from the 1940s to 
the 1960s and was responsible for at least 433 children with Dr Mary Barton stating that she had 
seen 600 prospective parents between 1944 and 195498497570, The Barton practice used donors 
from ‘intelligent stock’ and ruled out donors where there was ‘inheritable disease on that side’ or 
‘criminality’, introducing concepts of ‘the eugenic quality of the donor's stock’ into the very earliest 
years of the practice. While attempts were made to find suitable donors that were a ‘match’, 
(including parents choosing whether or not they wanted a ‘Jewish’ donor*”?)°”°, some early 
practitioners of artificial insemination used donors from ‘a very small panel of donors’*”, often from 
their own immediate social circles?””°”?. 

One prolific donor was Dr Barton’s husband, the scientist Bertold Wiesner, a consulting biologist at 
the Royal Northern Hospital in the 1940s. According to some estimates Wiesner may have 
fathered up to 1000 offspring during the time the clinic was operational’, despite a 1945 British 
Medical Journal paper where Barton and Wiesner stated they set an ‘arbitrary limit of 100 children 
for each donor’*°2. In the UK, a government register of donors was proposed as early as 1949*™, but 
such a register was not established until 1991%1. Subsequently, a number of people who have 
discovered they are the offspring of Wiesner have discovered each other by various means in 
subsequent years (including using direct-to-consumer genetic testing services) and formed an online 
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community. Some members of the group have disclosed their biological relation to Wiesner (and 
thus other siblings) in the public domain through media, including documentaries*26°’1>”3, and 
advocated for the rights of people who are donor conceived>”. 


Methods 
Case study data collection and analysis 


We used case study methodology to describe our experience of involving participants in an online 
discussion about genomics research together with pre and post discussion surveys. 


Case study selection 


The selection of this case study was informed by a number of factors which were appraised by the 
study team using the following questions?”*: 
e Was it a population of people affected by genomics research, distinct from the general 
public? 
e Was it pragmatic — was it possible to establish a mutually trusting and effective relationship 
within the time and resources of the research project? 
e Was the power dynamic equal and not exploitative (would the research offer participants 
something rather than just being passive subjects?) 
e Were there conflicting or competing interests which could negatively affect the research? 
e Was the proposed case study ethical (including a consideration of creating a capacity burden 
on populations or partner organisations) 


As part of a doctorate in public health genomics exploring public involvement in genomics research, 
one member study team (JN) began simultaneously planning a number of groups to work with to 
explore this area using participatory action research methodology. Unrelated to his PhD, he bought a 
direct-to-consumer DNA genetic test and subsequently discovered his grandfather was Bertold 
Wiesner and his mother was a half-sibling over up to 1000 other people’”®. After seeking advice from 
relevant ethics advisors, a proto-study team was assembled and began planning how to include the 
sibling group in co-designing a study to explore their views about involvement in genomics research. 
The study team contacted a researcher who had previously worked with members of this group 
(MC) and invited her to join the team in order to inform study design. The study team worked closely 
with both potential participants and experts from the La Trobe University human ethics department 
to ensure the method was acceptable and no one (including the study team) would be exposed to 
avoidable risk. 


Case study method 


The case study is presented as an instrumental case study, where the purpose is to understand the 
particular case and can attempt to provide data that could produce useful generalisations by using 
inferences from the data**’. The codesign of the case study was informed best practices for 
enhancing validity and rigour in the case study methodolgy14996423,176-178,368,370,418-420 The data 
collection and analysis was also informed by a number of frameworks for reporting involvement in 
research?”7987,179-181_ In addition to quantitative analysis, each source was analysed using the 
method of thematic analysis, which involved stages including data mapping and familiarisation, 
transcription, coding, searching for themes, reviewing themes with study team members, labelling 
and summarising themes and reporting the findings?”’. 

We collected and analysed both qualitative and quantitative data during the involvement activities. 
We also attempted to measure impacts, which can be outcomes from the participatory research 
process which have had an impact on individuals, the research process itself or wider society and 
other areas?7&1513 For example, participation in research might be shown to have a transformative 
impact on participants knowledge or views at an individual level, involving participants might have 
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an impact on the research design or learning from the research may change policy or 
practice®”192575.576, 

Two members of the study team were involved in analysing data from multiple sources including 
participant survey responses and online discussions (JN,MC). In addition, meeting notes, emails, 
surveys of the study team and reflexive diary entries of one member of study team (JN) were also 
analysed (JN). Coding and thematic analysis of qualitative data was carried out by two authors 
independently (JN, MC) and checked by another author (PL). Two authors of this paper also shared 
comments in the online study team discussion (JN, MC). Once a draft version of this article was 
created, it was shared with all participants who were invited to give feedback on the case study and 
contribute to STARDIT reports. 

An alpha version of the STARDIT framework was also used in parallel with the thematic analysis to 
organise data into pre-defined ‘super-categories’ which allow consistent comparison with other data 
using this reporting framework’, including other case studies. 


Study team survey 


The Facilitator (MC) was surveyed 6 months after the online discussion in order to integrate the 
valuable views and perspectives of those involved in planning and delivering the process. The survey 
questions can be found in the section ‘Facilitator survey questions’. Design of surveys was informed 
by best practice frameworks for public involvement’?®”. This method was informed by the Public 
Involvement Impact Assessment Framework Guidance (PiiAF)®” and the questions were informed by 
sections 7 and 8 of the GRIPP2 reporting checklist’. The Facilitator was asked 11 questions and the 
data was coded and categorised, including using the STARDIT framework 1°”. The data was then 
compared and integrated with the other data from the interviews and checked by other study team 
members (JN and PL). 


Investigator shared learning group 


During the online facilitation of the two online discussions, a shared learning group was established 
for facilitators and the study teams of two similar projects being run in parallel. The study teams 
shared reflections and learning about the process of facilitation online, as well as offering and 
receiving support regarding technical and practical issues. The data was coded and categorised, 
including using the STARDIT framework?™”. 


428 


429 


Survey questions 


Pre-discussion survey questions 


After participants had read the Participant Information and given consent to participate, they were 
asked to complete the following information. The questions below are worded exactly as 
participants read them. 


19 


20. 


21. 
22. 


23. 


24. 
25. 


26. 


27. 
28. 
29. 


30. 


31. 
32. 


33. 
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Full Name (if you would prefer to not use your real name you may use a pseudonym) 


Email address (Please note this needs to be a working email address. If you do not have one 


leave this blank and we will contact you by your preferred method of communication.) 
Phone number (optional) 
Any other preferred method of communication? (Please share any other preferred method 
of communication if email or phone are not preferred) 
Age 
Gender (Choose from ‘Male’, ‘Female’, ‘Transgender’, ‘Intersex’, ‘Other’, ‘Prefer not to say’) 
Educational background (tick all that apply) 
a. Middle school qualifications (up to age 16) (‘lower’) 
b. High school qualifications (ages 16-19) (‘middle’) 
c. Degree (bachelors), diploma or post-graduate (‘higher’) 
d. Ihave qualifications or professional experience in genomics (professional) 
e. Prefer not to say 
Please tick which of the following statements that you agree with: 
a. | feel comfortable describing other descendants of my biological father as ‘half- 
siblings’ 
b. | would describe our email group as an ‘online community’ 
c. Members of this email group potentially have a shared interest in discussing future 
research which might affect them, including genomic research 


d. If you do not feel comfortable describing other descendants of your biological father 


as 'half-siblings' please share any term (or terms) you prefer. 
In which country do you live (or spend most time)? 
What made you decide to respond to our invitation to participate in this project? 
What do you hope to get out of participating in this discussion? Do you have any specific 
expectations? 
There are many benefits of involving people other than researchers in the co-design of 
research studies at every stage of the research cycle. Research suggests that involving 
people improves the quality and the relevance of the research. Involving people can also 
improve participant experience and increase participation. Who do you think should 
influence what kind of genomic research should be done in the future? 
What makes you say that? (why did you give that answer?) 
Do you have any ideas about how the people from your previous answer could influence 
future research? (For example, what tasks could people affected by EGID be involved in?) 
Which aspects of any future research genomic research should be influenced by the 
following (participants were presented with a grid of tick boxes, the horizontal axis being 
who should be involved, the vertical a list of tasks. The horizontal was as follows) 


Everyone Anyone who | Only people | Only people Only people 
(any member | might be who are who are with a 
of the public | indirectly directly participating | professional 
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who is affected by affected by inthe role in 
interested) the research | the research | research research 
a. All aspects mentioned below (leave others blank if ticking this) 
b. Finding questions to ask (identifying research topics) 
c. Deciding which questions to prioritize and fund 
d. Deciding how to try and answer the question (the research method) 
e. Attempting to answer the question (carrying out the research, including collecting 
information) 
f. Trying to understand if it is possible to answer the question (analyzing the 
information) 
g. Sharing the information that has been found, and any answers that may have 
emerged (dissemination and publication) 
h. Ensuring that any information or answers are able to be used to help people in 


j. 


practice, policy or future research (sometimes called research translation) 
Deciding if the way of asking the question and all the other stages of the research 
were appropriate (evaluating the research method and any impacts) 

Designing how people are involved in the research 


34. Have you ever participated in research in the past? (by participation, we mean as a research 
subject — for example part of a trial) 


a. 
b. 


C. 


d. 


Yes 

No 

Prefer not to say 
Unsure 


35. Have you ever participated in research in the past? (by participation, we mean as a research 
subject — for example part of a trial) 


a. 


b. 
C. 
d 


Yes 

No 

Prefer not to say 
Unsure 


Post-discussion survey questions 
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22. How would you rate the following? (chose from ‘Excellent’, ‘Somewhat goog’, ‘Neither good 
nor bad’, ‘Somewhat poor’, ‘Extremely poor’) 


a. 


b. 


d. 


Your overall experience of participating in the online discussion 
Your assessment of how we conducted the survey and discussion format 


The support you received to be involved (for example, practical support such as 
instructions for using the online tools)? 
Information and learning materials you were given before the event 


23. Did you feel you meaningfully contributed to the discussion? 


a. 
b. 
C. 


Yes 
No 
Unsure 


24. Is there anything in particular you liked or thought was helpful about how the discussion was 
conducted? 

25. Is there anything you didn’t like, thought was unhelpful. or could have been improved about 
how the discussion was conducted? 


26. 
27. 


28. 
29. 


30. 


31. 
32. 


33. 
34. 


35. 


36. 
37. 
38. 
39. 
40. 
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Do you have any other thoughts, ideas or comments? 

Would you like to be updated about the progress of the research and offered chances to be 
involved where possible? (Choose ‘yes’ or ‘no’) 

Did you have any expectations from participating in this research that were met or not met? 
Have any of your views and perspectives about involving people in genomic research 
changed since participating in this research? If so, please describe. 

There are many benefits of involving people other than researchers in the co-design of 
research studies at every stage of the research cycle. Research suggests that involving 
people improves the quality and the relevance of the research. Involving people can also 
improve participant experience and increase participation. Who do you think should 
influence what kind of genomic research should be done in the future? 

What makes you say that? (why did you give that answer?) 

Do you have any ideas about how the people from your previous answer could influence 
future research? 

For example, what tasks could people be involved in? 

Which aspects of any future research genomic research should be influenced by the 
following (participants were presented with a grid of tick boxes, the horizontal axis being 
who should be involved, the vertical a list of tasks. The horizontal was as follows) 


Everyone Anyone who | Only people | Only people Only people 
(any member | might be who are who are with a 

of the public | indirectly directly participating | professional 
who is affected by affected by inthe role in 
interested) the research | the research | research research 


All aspects mentioned below (leave others blank if ticking this) 

Finding questions to ask (identifying research topics) 

Deciding which questions to prioritize and fund 

Deciding how to try and answer the question (the research method) 

Attempting to answer the question (carrying out the research, including collecting 

information) 

f. Trying to understand if it is possible to answer the question (analyzing the 
information) 

g. Sharing the information that has been found, and any answers that may have 
emerged (dissemination and publication) 

h. Ensuring that any information or answers are able to be used to help people in 
practice, policy or future research (sometimes called research translation) 

i. Deciding if the way of asking the question and all the other stages of the research 

were appropriate (evaluating the research method and any impacts) 
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j. Designing how people are involved in the research 
Full Name (Optional- if you would prefer to not use your real name you may use a 
pseudonym) 
Email address (optional) 
Phone number (optional) 
Age 
Gender (Choose from ‘Male’, ‘Female’, ‘Transgender’, ‘Intersex’, ‘Other’, ‘Prefer not to say’) 
Educational background (tick all that apply) 
a. Middle school qualifications (up to age 16) (‘lower’) 
b. High school qualifications (ages 16-19) (‘middle’) 


41. 


42. 
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c. Degree (bachelors), diploma or post-graduate (‘higher’) 
d. Ihave qualifications or professional experience in genomics (professional) 
e. Prefer not to say 
Please tick which of the following statements that you agree with: 
a. | feel comfortable describing other descendants of my biological father as ‘half- 
siblings’ 
b. | would describe our email group as an ‘online community’ 
c. Members of this email group potentially have a shared interest in discussing future 
research which might affect them, including genomic research 
d. If you do not feel comfortable describing other descendants of your biological father 
as 'half-siblings' please share any term (or terms) you prefer. 
In which country do you live (or spend most time)? 


Facilitator survey questions 


The Facilitator (MC) was surveyed 6 months after the online discussion in order to integrate the 
valuable views and perspectives of those involved in planning and delivering the process. Design of 
surveys was informed by best practice frameworks for public involvement””®’. This method was 
informed by the Public Involvement Impact Assessment Framework Guidance (PiiAF)*®” and the 
questions were informed by sections 7 and 8 of the GRIPP2 reporting checklist’. The Facilitator was 
asked 11 questions and the data was coded and categorised, including using the STARDIT framework 
192 The data was then compared and integrated with the other data from the interviews and 
checked by other study team members (JN and PL). 


14. 
15. 


16. 


17. 


18. 


19. 


20. 


21. 


22. 


23. 


24. 


25. 


26. 
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Please describe your tasks in the process of involving people in planning of the study 
What did you learn from the process of involving participants in the research planning 
phase? 

Please describe specifically what worked well or was useful about the way the study was 
conducted (including how people were involved) 

Please describe specifically what did not work well or was not useful about the way the 
study was conducted (including how people were involved) 

Were there any barriers or enablers to conducting the study or involvement activities? 
(institutional or otherwise) 

Do you think the involvement activity achieved its intended aim(s)? 

Do you think the study achieved its intended aim(s)? 

Do you have any advice to other researchers planning involvement for their research? 
Do you have any advice to other researchers planning to involve people using online 
discussions? 

Describe the impact you think involving people had (positive/negative - on the research, 
staff or participants) 

Who do you think should influence the kind of human genomic research done in the future, 
and why? (e.g. the public, participants of research studies, doctors, school children, 
politicians etc) 

Which stages of future genomic research should be influenced by people other than 
researchers (if any)? (e.g. concept planning of new studies, study design, conducting the 
research, presenting the results etc) 

Other comments 
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Learning resources 


A number of different learning resources were shared with participants at different stages of the 
process. This included a short 60-second online video about the study, giving information about the 
context and purpose??, a one page infographic summary of a scoping review about genomics 
research?®8, 


Learning resources were both co-created and selected by the study team, working in partnership 
with the Australian Genomics Health Alliance and co-refining the selection with potential 
participants. In addition, in order to support the facilitators in providing good quality information, a 
number of were curated into a list to be available for facilitators to share during the online 
discussion, if they became relevant to aspects of the discussion in order to help inform people. The 
next section summarises the resources used, with references using Internet Archive links to future- 
proof the content of the learning resources as well as the URL. 
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Summary of Learning Resources 
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Stage Title Media Summary Authorship 
Stage 2: What is genomic Portable Simple infographic Australian 
Before testing? Document explaining the basics of Genomics Health 
consent Format (PDF) | genomics research in plain | Alliance®® 
English 
What is genomics? | Online video A 6-minute video outlining | Genome BC>*° 
animation the principles of genomics 
with audio research 
narration and 
subtitles 
Stage 3: Definitions and PDF These definitions and Study team 
Before online | explanations explanations were used as_ | (see ‘Learning 
discussion a glossary to explain the resource example 
main concepts of this 1’ below) 
research project. 
Genomics and Online video A 2-minute video Jack Nunn?°” 
involvement with hard- exploring why people 
coded text should be involved in 
genomics 
Infographic PDF A one page summary of Jack Nunn et al?88 
summary of (infographic— | the main findings from a 
scoping review images and recent scoping review 
text) about involving people in 
genomics? 
Guide to using PDF (text with | A co-created learning Study team 
Loomio hyperlinks) resource giving practical 
advice for using the online 
discussion platform 
Loomio 
Additional Inheriting genetic Webpage An additional resource if U.S National 
resources conditions participants wanted more | Library of 
available to (chapter) information about Medicine*® 
facilitator inherited conditions 
Data in the 100,000 | Online video An example of the Genomics 
Genomes Project animation bioinformatic pathway England? 
with voice (specific to Genomics 
over and England) but generalisable 
subtitles (talks about access review 
committee) 
Ethical issues in PDF (text with | Additional resource for a Centre for 
human hyperlinks) relevant ethics discussion Genetics 
genetics and from a medical Education*® 
genomics perspective 
Genes, DNA and Webpage Good plain English Cancer Research 
cancer (text with information about genes UK 
hyperlinks) in relation to cancer. 
How to Share Webpage Good information about American Society 
Genetic Test (text with sharing genetic test results | of Clinical 
Results With Family | hyperlinks) with family. Oncology>*” 
How do you Image file Infographic about the Genomics 
sequence ahuman_ | (infographic— | stages of genome England”®? 
genome? images and sequencing 
text) 
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Learning resource example: Definitions and explanations 


The following definitions and explanations were used to explain the following concepts throughout 
this research project. This document was formatted as a PDF with references. 

Genomics 

The study of all the DNA in the genome together with the technologies that allow it to be 
sequenced, analysed and interpreted is collectively called genomics, or genomic medicine if applied 
to patients**. The study of genomics can include other types of “omics”*, such as ‘proteomics’ and 
‘metabolomics’ — which for simplicity will be referred to under the term ‘genomics’. When the term 
‘genomics’ is used in this project, it refers exclusively to human genomics. 

*Other types of “Omics” include transcriptomics (all the RNA molecules in cell or organism), 
proteomics (all the proteins in a cell or organism) and metabolomics (all the metabolites in a cell or 
organism). 

Genomic research 

Genomic research refers to any kind of activity which is intended to increase our current 
understanding of genomics. It is distinct from genomic medicine or other routine services which use 
existing knowledge, rather than add to it. However, this distinction is not always clear. 
Involvement in genomic research 

Research should be conducted ethically and to benefit people. Involving people as equal partners in 
genomic research has been identified as the most crucial aspect, as the benefits include improved 
public trust 2°. The concept of ‘public involvement’ in research is defined as research that is carried 
out ‘with’ people rather than ‘on’ them 2°. This allows people to have ‘active’ rather than ‘passive’ 
roles in research, which can lead to better research outcomes. Involving people in research in this 
way is now promoted by many governments as a right and is predicted to increasingly be an 
obligation in biomedical research, with research funding initiatives already recognising evidence of 
involvement as a key criterion 3312577, Involving people can give greater public influence over 
research directions and conduct *4, ensuring research is both acceptable, accessible and meets 
people’s needs by reflecting and balancing the diversity of priorities 18337, Tasks that people can be 
involved in include identifying areas of benefit, helping design and plan studies, helping in raising 
funds and analysing results. 

This project 

This project is part of Jack Nunn’s PhD, ‘Genomics research and involving people’ which is focused on 
exploring how we can better involve people in all stages of genomics research, including the best 
methods to do so. 
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Data 


Data sources 
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This table summarises all the data sources used for the case study. 


Data Category 


Data point description 


Diary 


Research diary of lead investigator (JN) — including reflections 
during the process 


Emails and meeting notes 


Email, meeting notes and notes from planning and discussion. This 
included involvement of potential participants in co-designing and 
co-refining the study. 


Online pre-discussion survey 


Informed consent and pre-discussion survey data 


Learning resources 


Learning resources for participants and the Facilitator giving 
information about genomics and using Loomio (see section 
‘Learning resources’) 


Online discussion with 
participants 


Text data from online Loomio discussion with participants 


Online discussion with facilitators 


Text data from online Loomio discussion between facilitators of 
two parallel studies 


Online post-discussion survey 


Post-discussion survey data from participants 


Follow up survey for facilitators 


Post-discussion survey data from facilitators and additional emails 
with further reflections 
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STARDIT Preference Mapping (STARDIT- PM) 


This table uses the Alpha version of the Standardised Data on Initiatives Preference Mapping 
(STARDIT-PM) to categorise the data into certain areas’**. Preferences were recorded from all data 
sources, including the initial survey, online discussion with participants, online facilitator discussions, 
follow-up surveys with participants and with facilitators. Facilitator comments in the online 
discussion were not included. If the same participant made the same point at different stages, this 
was counted as one view. The standardised categorisation is intended to facilitate comparison with 
other studies. Accordingly, the content may be similar to other sections of the qualitative thematic 


analysis. 
STARDIT-PM area 
and quantitative 
data 
Views on who should 
be involved: 


10 participants 
shared views about 
who should be 
involved 


437 


Qualitative summary 


Participants stated that anyone should be involved in research, with a 
experts, people affected by the research directly and the public all 
sharing perspectives in the context of research carried out with ethical 
oversight. Six participants stated that anyone should be involved in 
research, with one participant stating ‘everyone should have a voice not 
just scientists and researchers’ [P5]. Another participant stated ‘it needs 
to be a wide-ranging discussion so that the benefits and possible 
problems can be fully explored’ [P4]. One participant said it can depend 
‘what kind of research it is’ and what the purpose is [P7], with another 
adding ‘we all need to have a voice’ as ‘we may not be “experts” in 
genomics but our opinions must be respected and have validity’ [P5]. 


Six participants stated those affected by research should be involved, 
however others challenged this saying this could provide ‘a rather one- 
eyed perspective’ [P4]. One participant stated ‘some research will 
benefit certain people - those people should probably influence it if 
they are an identifiable group’. Another participant noted that ‘we're all 
biased; whoever is affected by a condition is likely to want it 
prioritised’[P12]. Another participant concluded ‘we're all biased; 
whoever is affected by a condition is likely to want it prioritised’[P12]. 


One participant stated that people (including the public and research 
participants) will have a ‘variety of professional and technical and 
creative skills’ which will be useful, with the most useful one being 
‘knowing ourselves’ [P7]. 


One participant stated ‘I am a strong supporter of patient involvement 
in medical care’ and that ‘involving members of the public’ in genomic 
research was important in order to ‘have their views, reactions, 
interpretations, questions, concerns sought, interacted with, and 
considered’ [P11]. 


Two participants stated that experts (including ‘scientists’[P9]) who 
‘know what they are doing’ should be involved [P2], with ‘research 
reviewed by ethics boards’ [P9]. 


There was a recognition that different groups in society might have 
different interests and influence. One participant articulated groups 
including ‘medical scientists’, ‘social scientists’, ‘psychologists’ and the 


Views on specific 
tasks people 
involved could do: 


8 participants 
shared views about 


specific tasks people 


involved could do 


Views on modes of 
communication: 


4 participants 
shared views about 
preferred 
communication 
modes 


Views on what 
methods should be 
used to involve 
people: 
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general public as being groups which should influence research, but 
noted that not all ‘groups should have equal influence’[P10]. One 
participant asked ‘there will be many interested groups so which ones 
will be listened to?’ [P4] One participant stated that ‘people who are 
not looking for personal gain, but who have a desire to improve quality 
of life and help us understand ourselves’ should influence research [P6]. 
One participant said that research participants should be involved in 
‘agreeing purpose, parameters and methods’ [P7]. Another asked 
‘whatever format is decided upon who would decide on the points for 
discussion?, implying that participants’ tasks should include deciding 
this [P4]. One participant added that it is a ‘good idea to involve 
research subjects in formulating the research questions’ [P10]. Another 
participant stated they should be involved in ‘having a say in what 
research is supported by public money’ and ‘making sure that the uses 
and purposes to which the research is put are responsible and allied 
with the laws and mores of our society’ — which includes ‘ethical 
oversight’ [P7]. One participant also added that the public should have 
a voice in how ‘science and research can better involve’ people [P5]. In 
reference to future research with the sibling group one participant 
stated that ideally ‘we would be able to exert control over the use’ of 
data [P7]. One participant felt they should be involved in ‘seeking 
answers to old, or not yet thought of questions’ and ‘looking beyond 
the known into a murky unknown’ [P6]. The discussion also explored 
who should be involved and in which tasks. One participant noted they 
didn’t feel ‘qualified’ to ‘comment on aspects of science itself’ but felt 
‘strongly’ that they should be involved in ethical decisions and sharing 
personal experiences to help inform research [P5]. They also stated 
experts ‘need to drive research’ but they ‘cannot do it in vacuum’ as the 
public need them and they need the public [P5]. 

One participant stated ‘moderated face to face discussions (through 
video if need be) remain the best method in my opinion for focussed 
outcomes and decisions with groups of people’ [P7}. Another 
participant stated that ‘most of our group would be able’ to use video- 
conferencing platforms [P5]. Face-to-face synchronous discussion was 
ruled out by another participant as there are ‘too many voices’ which 
are across multiple time-zones [P4]. Communicating via a verbal 
interview was perceived as taking less time than anything which 
‘requires a lot of writing’ [P4]. One participant said they were ‘happy to 
contribute in any way in which is practical online’ [P5], with another 
adding ‘email is still the best way’ [P4]. One participant suggested there 
would be ‘differing preferences for how such a project should be 
organised’ and suggested agreeing on some ideas (not using email) and 
then sending these ideas to the group once decided [P4]. One 
participant concluded that ‘a forum for considered comments’ online 
can be useful as long as enough time is allowed [P7]. 

The participatory research method was described as ‘commendable’ 
[P9]. Participants suggested the idea of using one to one interviews as 
way of involving people (including using telecommunications) [P5], 
however, one participant noted that one-to-one interviews can restrict 
discussion and ‘be quite straight jacketed with circumscribed 
questions’, compared to online discussions [P4]. A ‘group of “special 


5 participants 
shared views about 
what methods 
should be used to 
involve people 


Views on enablers 
(facilitators) of 
involvement: 


7 participants and 3 
facilitators shared 
views about 
facilitators of 
involvement 
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interest” people involved in a group discussion’ was suggested as a 
‘simple but effective method of encouraging debate’, if participants can 
‘can dip in and out’ [P4]. They also stated group discussion would not 
work as ‘there are too many voices and some would be drowned out’ 
[P4]. This participant also stated ‘we will have differing preferences for 
how such a project should be organised’ and asked ‘would it be possible 
to agree on some ideas and then post them to the group’ in order to 
involve people in co-creating how they will be involved and participate 
[P4]. 


One participant suggested that discussion, interviews, surveys and 
‘documents and videos shared for feedback’ would all be viable 
methods of involving people [P5]. 


Another participant suggested a professional ‘market researcher’ who 
was ‘tasked with finding a cross-section of people’ might be a helpful 
method to involve people [P4]. 


One participant shared highly-specific views about the method that 
should be used, stating ‘It should not be a plebiscite’ nor ‘self-electing 
moral Praetorian guard’, adding that ‘the more diverse the debate, the 
more dilute the effect of irrational preconception and ethical 
incompetence should become. The model of representative democracy 
seems to me the best available’ [P8]. 


Another participant stated that ‘ongoing discussions using social media 
and specific pages’ such as Facebook pages, could be a good method to 
involve people [P5]. 


Another participant concluded that ‘Moderated face-to-face discussions 
(through video if need be) remain the best method in my opinion for 
focussed outcomes and decisions with groups of people’, adding that ‘a 
forum for considered comments which are neither binding nor meant 
to be conclusive such as this can certainly be online’ as long as enough 
time is allowed’ [P7]. 

One participant noted that being ‘highly educated’ was an enabler for 
involvement and that having a ‘bit of time on their hands’ was also an 
enabler [P4]. Being ‘respectful’ when involving ‘those affected by 
genomic research’ will facilitate research as the ‘more brains applied to 
research, the more likely answer to puzzles will be found’ [P11]. 
Similarly, another participant stated ‘the more diverse the debate, the 
more dilute the effect of irrational preconception and ethical 
incompetence should become’ [P8]. One participant stated that 
whatever model was chosen, it should be ‘as flexible as possible’ [P5]. 


Four participants reported specific things about the way this study was 
conducted that facilitated their involvement. One participant said the 
entire process was ‘assiduous’ and that the ‘intent of this project’ was 
‘obviously thoughtful and interesting’ [P9]. One participant said the 
‘system seemed to work well’ [P7]. Another added that being used to 
online platforms like Loomio, or having previous experience of similar 


Views on barriers of 
involvement: 


6 participants and 3 
facilitators shared 
views about barriers 
of involvement 
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platforms and ‘used to’ that way of communicating might facilitate 
involvement using that communication mode. One participant 
suggested an alternative discussion format where the participants 
discussed a thread for 2 days and then had a 3 day break before coming 
to another thread [P7]. 


The Facilitator (MC) noted that regular contact with the study team and 
timely support was essential and they ‘could not have done it without 
this’. 


Barriers to involvement in research identified by participants included 
public fear and ‘hysteria’ caused by a lack of understanding, which may 
‘hamper’ research, involvement and general public support for research 
[P5]. Synchronous discussion was highlighted as another barrier if 
participants ‘are across time zones’ [P4]. One participant mentioned 
that they felt that their emotional response to some issues made it 
difficult to get involved in some ways [P6]. Being required to watch 
lengthy videos was identified as a barrier by one participant. One-to- 
one interviews were mentioned as being ‘quite straight-jacketed with 
circumscribed questions’ compared to more open online discussions 
[P4]. They also stated ‘I don't think that a group discussion would work 
as there are too many voices and some would be drowned out’ [P4] 


Four participants reported specific things about the way this study was 
conducted that were barriers to their involvement. A discussion about 
boundaries revealed that some participants felt ‘avoiding topics which 
might trigger emotions which are stressful or unpleasant’ could be 
viewed as ‘restrictive, even censorious’ [P7]. The pace of the 
discussions was mentioned as moving ‘too quickly’ with another adding 
‘more time’ was needed and study team should ‘reconsider the pace of 
the research’ [P7] [P4] [P5]. Updates from the discussion were sent to 
participants according to their preferences, and one stated they ‘lost 
track of emails’ and were sometimes unsure if they were ‘responding to 
the right part’ [P6]. Two participants stated the ‘platform presented 
technical difficulties’ [P4] and that it was ‘complicated’ [P5]. One 
participant stated the ‘premise and the purpose of the study could be 
clearer’ and that the various discussion threads were ‘difficult to 
untangle sometimes’ [P7]. They also mentioned it was ‘hard to be able 
to guarantee to do this every day for a period’ and that not doing so 
meant they ‘got lost’ [P7]. Another participant added that ‘it’s a difficult 
subject to discuss in a vacuum, without real life examples’ [P4]. One 
participant expressed ‘trepidation’ at sharing views about research and 
compared the feeling to getting an answer wrong in an ‘exam’ [P6]. 


The facilitator (MC) stated that they felt more time was required in the 
co-design process. In addition, the administrative processes 
surrounding the unplanned change of Chief Investigator and related 
administrative processes in relation to the ethics process (outside of the 
control of the study team) meant they felt support was ‘non-existent’ 
and was ‘wholly inadequate’ for the participatory research process 
being used. 


Views on what the 
outcome or output 
of the involvement 
could be 


2 participants 
shared views on 
what the outcome or 
output of the 
involvement could 
be 

Views on which 
stage of the research 
people should be 
involved: 


1 participant shared 
views about which 
stage of the research 
people should be 
involved 

Views on who should 
the data from this 
project shared with?: 


3 participants 
shared views on who 
data from this 
project should be 
shared with 

Views on how think 
learning from this 
research could be 
used 

8 participants 
shared views about 
how learning from 
this research could 
be used 
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One participant stated they wanted to know that their involvement had 
been ‘useful to the researchers’ [P11], with another stating an outcome 
of being involved would be the ‘satisfaction of knowing that | may have 
contributed’ [P10] 


One participant noted that changing study design in the later stages of 
a study not always possible, posing the question ‘can we be both 
subjects and supervisors - at the beginning yes, later, maybe not’ [P7]. 
The participant also stated that compared to ‘highly-qualified scientists 
who know what's possible’ some ‘relatively ignorant lay people’ will be 
limited in what they can offer at some stages [P7]. The participant 
suggested that getting the ‘purposes, the parameters and the methods 
agreed with all participants’ at the design stage might be most 
appropriate [P7]. 


Participants shared views about who data should be shared with, with 
One participant stated explicit concerns about sharing data for political 
or financial use [P5], with other participants agreeing. One participant 
stated ‘Research is for humankind. Its benefits should be available to 
all. Information should be for the most part easily available. If it's 
publicly funded, it must be publicly available [P9]’. Cultural conventions 
around ownership of knowledge such as ‘patenting’ were challenged as 
forms of knowledge control which are ‘unethical’ in some contexts. 


Eight participants shared multiple views on how learning from a 
proposed study could be used, including predicting human traits (for 
example, risk of diseases or mental health problems). One participant 
indicated the study could help individuals in the group understand 
‘what unwelcome genes we might have inherited’ [P4]. Participants also 
indicated the research could be used to improve understanding of how 
things like personal experience, socioeconomic circumstances and 
culture interact with genomics. One participant added ‘longitudinal 
studies might well be set up to study a cohort of babies with certain 
genomic sequences which predispose them to certain diseases’ [P4]. 
Another participant stated research with the group ‘could have 
implications for all’ people [P3]. 
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Demographic information 


Pre-discussion Post discussion 


Category survey Survey 

Gender 

Female 7 3 
Male 5 3 
Age 

50-54 years 1 0 
55-59 years 2 0 
60-64 years 0 0 
65-69 years 6 4 
70-74-years 3 2 
Educational background (‘highest' only counted) 

Degree (bachelors), diploma or post-graduate (‘higher’) 11 5 
High school qualifications (ages 16-19) 1 1 
Number who agreed with following statements 

| feel comfortable describing other descendants of my 12 N/A 
biological father as ‘half-siblings’ 

| would describe our email group as an ‘online community’ 9 N/A 
Members of this email group potentially have a shared 

interest in discussing future research which might affect 11 N/A 
them, including genomic research 

In which country do you live (or spend most time)? 

Canada 3 2 
Greece 1 0 
Spain 1 1 
United Kingdom 7 3 
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Views about who should be involved in research 


Participants were asked the same questions before and after the online discussion. When asked who 
should be involved in various tasks in research, participants could choose from the categories 
outlined in Figure 2 in the main article. A change in direction is described as ‘widening’, the inverse 
as ‘narrowing’. Widening was calculated as being a move towards an attitude that more people 
should be involved in research, whereas narrowing was calculated as a move towards an attitude 
that fewer people should be involved. 

A total of 54 responses were given by 6 participants where participants completed answers to 
questions of both the baseline and follow-up survey. 35% of responses showed a change towards 
‘widening’ involvement (N=19/54) while 8% ‘narrowed’ (N=8/54). 50% of responses stayed the same 
(N=27/54). 


Widening and narrowing for each question 


Who should influence which aspects of research? | Change No change | Change to 


to wider narrower 
Finding questions to ask 2 4 0 
Deciding which questions to prioritize and fund 2 3 1 
Deciding how to try and answer the question (the | 2 2 2 
research method) 
Attempting to answer the question (carrying out 2 3 1 
the research, including collecting information) 
Trying to understand if it is possible to the answer | 3 3 0 
the question (analysing the information) 
Sharing the information that has been found, and | 2 3 1 


any answers that may have emerged 
(dissemination and publication) 
Ensuring that any information or answers are able | 2 3 1 
to be used to help people in practice, policy or 
future research (sometimes called research 
translation) 

Deciding if the way of asking the question and all 2 4 0 
the other stages of the research were appropriate 
(evaluating the research method and any impacts) 
Designing how people are involved in the research | 2 2 2 


Change total Dp? 
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Participant experience 


A 


Somewhat good 
Neither good nor bad 


Excellent 1 
Somewhat good 2 
Neither good nor bad 1 
Somewhat poor 1 


Excellent 2 
Somewhat good 2 
Neither good nor bad 1 


w 


Excellent 
Neither good nor bad 


N 


Yes 3 
Unsure 


w 


Investigator shared learning group 


During the online facilitation of the two online discussions, a shared learning group was established 
for facilitators and the study teams of two similar projects being run in parallel. The study teams 
shared reflections and learning about the process of facilitation online, as well as offering and 
receiving support regarding technical and practical issues. The data was coded and categorised, 
including using the STARDIT framework?”. 
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Data analysis 
Quantitative analysis of all themes 


Theme 


Research for profit and ‘Bullying’ by ‘big pharma’ 

Anyone should be involved in research 

Those affected by research should be involved 

Research with sibling group is unique and complex but 
important 

Who decides who decides what is ethical 

Concerns about genomics research being used for political 
purposes 

Finding out they are part of sibling group has been a 
positive experience 

View on topics for research 

Participants reported changed views and perspectives as a 
result of participating 

Desire to improve situation for people affected by assisted 
conception 

Interested in learning what other siblings think and discuss 
issues 

Concerns about control of knowledge and data 
Questioning which groups should have ‘equal influence’? 
Questioning eugenic attitudes to genomic variations 
Views on participation in genomics research 

Participants learned about genomics 

Motivation for participation to help researchers and sibling 
group 

Uncertainty about what they can offer but happy to help 
What is the purpose of research? 

Experts should be involved (over seen by ethics boards) 
Developments in genomics have significant implications for 
society 

People have responsibility to be involved in research 
Questioning giving power to experts 

What control do research participants have? 

People with specific experience and skills should be 
involved 

The public and research participants should be involved in 
research 

How should people be involved in genomics research 
Tasks of involvement 

Stages of involvement 

Questions genomic medicine as intervention before other 
methods 

Questioning genomics determinism 

Choosing what to know about your genome 
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Qualitative data analysis 
Emergent themes from qualitative participant data 


This section organises qualitative data from participants into themes. This analysis has been 
provided in order to improve transparency of the qualitative analysis process which is summarised in 
the accompanying article. 


Detailed Summary of Thematic Area 1: Participant views about involvement in genomic research 


Participants demonstrated an understanding of the difference between participation in research and 
involvement in research. One participant stated ‘I am a strong supporter of patient involvement in 
medical care’ and that ‘involving members of the public’ in genomic research was important in order 
to ‘have their views, reactions, interpretations, questions, concerns sought, interacted with, and 
considered’ [P11]. Participants explored ideas around the purpose of research and one stated ‘if | 
was the researcher running the project, | would want to get the purposes, the parameters and the 
methods agreed with all participants’ [P7]. One participant also added that the public should have a 
voice in how ‘science and research can better involve’ people, which aligns with best-practice 
identified in a review of public involvement in genomics research [P5]°”°. 

Six participants of the online discussion stated that anyone should be involved in research, with one 
participant stating ‘everyone should have a voice not just scientists and researchers’ [P5]. Another 
participant stated ‘it needs to be a wide ranging discussion so that the benefits and possible 
problems can be fully explored’ [P4]. One participant said it can depend ‘what kind of research it is’ 
and what the purpose is [P7], with another adding ‘we all need to have a voice’ as ‘we may not be 
“experts” in genomics but our opinions must be respected and have validity’ [P5]. 

Methods of involving people were discussed in detail with a number of options explored. 
Participants suggested that discussion, interviews, surveys, ‘representative democracy’ and 
‘documents and videos shared for feedback’ would all be viable methods of involving people [P8] 
[P5]. The participatory research method was described as ‘commendable’ and some participants 
suggested the idea of using one to one interviews as way of involving people (including using 
telecommunications) [P9] [P5], however, one participant noted that one to one interviews can 
restrict discussion and ‘be quite straight jacketed with circumscribed questions’, compared to online 
discussions [P4]. This participant also stated ‘we will have differing preferences for how sucha 
project should be organised’ and asked ‘would it be possible to agree on some ideas and then post 
them to the group’ in order to involve people in co-creating how they will be involved and 
participate [P4]. Another participant concluded that ‘a forum for considered comments which are 
neither binding nor meant to be conclusive such as this can certainly be online’ as long as enough 
time is allowed [P7]. 

Participants used a number of literary references to frame ethical debates, including citing The 
Tempest by Shakespeare as offering helpful analogies in exploring the ethics of the ‘brave new 
world’ of genomic research [P9]°”, with one participant stating ‘I feel more like a mix of Ariel or 
Caliban with respect to research - willing to help (and hoping not to be enslaved) but also willing to 
cede leadership’ [P5]. Public fear and ‘hysteria’ caused by ‘Brave New World fantasies’ and a lack of 
understanding may ‘hamper’ involvement and general public support for research. [P9] Another 
participant disagreed, adding ‘if we say nothing, do nothing that has to be worse doesn't it?’ [P5]. 


One participant noted that changing study design in the later stages of a study not always possible, 
posing the question ‘can we be both subjects and supervisors - at the beginning yes, later, maybe 
not?’, recognising that involvement at the design stage is the most practicable, which aligns with 
other similar studies [P7]®. 


Six participants stated those affected by research should be involved, however others challenged 
this saying this could provide ‘a rather one eyed perspective’ [P4]. One participant stated ‘some 
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research will benefit certain people - those people should probably influence it if they are an 
identifiable group’. Another participant noted that ‘we're all biased; whoever is affected by a 
condition is likely to want it prioritised’ [P12]. Being ‘respectful’ when involving ‘those affected by 
genomic research’ will facilitate research as the ‘more brains applied to research, the more likely 
answer to puzzles will be found’ [P11]. One participant said that research participants should be 
involved in ‘agreeing purpose, parameters and methods’ [P7]. 

Six participants expressed concern about research for profit and those with financial interests 
influencing research. One participant stated they would be happy to participate in research but that 
they ‘would however be concerned if this data was ever shared for commercial purposes’ [P5]. 
Another participant noted they felt that certain pharmaceutical companies were responsible for 
‘bullying’, contributing to ‘disinformation; ignorance and inflexibility of medical and scientific 
professions’ [P12]. Another participant raised concerns about ‘sponsored facts’ and asserted that 
the opinions about genomics research ‘must be respected’ [P5]. Another participant stated that ‘the 
people paying for the research will influence it’, noting that if it is ‘public money the gatekeepers will 
have the greatest influence’ [P7]. One participant said they find it ‘distasteful that a private company 
can benefit from public research and then withhold the data or charge for it’ [P9]. 

While some participants felt ‘trust’ towards existing scrutiny for research oversight [P5], ethics 
committees were identified as having ‘failings’, and one participant asked the question ‘who will 
decide who will be on the ethics committee?’ [P4]. Another participant added ‘I am not sure of the 
ethics process but it does seem a shame that more of us cannot participate’ [P5]. Another 
participant stated ‘I trust the scientists and the ethical committees’ [P9], while another noted that 
‘the ethics of DNA research generally will continue to be of huge importance and will continue as a 
political issue, triggering new laws and regulations’ and raised a concern that ‘the law will not be 
able to keep up with the research - and we as members of the general public won't either’ [P7]. One 
participant noted that while people might be experts about a process or data, but that does not 
make them ‘moral guardians’ [P5]. They concluded ‘no single body (and that includes the church and 
the government) has a right to dictate moral guidelines’ [P5]. 

Participants indicated that all health policy decisions have political power associated with them, and 
five participants raised a number of specific concerns about genomic research being mis-used for 
political purposes and ‘wicked ends’ [P5]. One participant stated research should not be used ‘for 
political purposes’ but indicated they did not believe this was easily prevented [P5], with another 
participant sharing the ‘realism about the likely corrupt use of genetic information for political and 
financial purposes’ [P2]. 

Multiple participants agreed that the public can get involved in publicly funded research by 
overseeing what research is funded (identifying and prioritising) and ethical oversight [P7] [P5]. 
Three participants challenged established forms of knowledge and data control, stating that if 
research is ‘publicly funded, it must be publicly available’ and challenging concepts around 
intellectual property and patent laws, and that ‘patenting new life forms is unethical’ [P9]. Another 
participant stated they should be involved in ‘having a say in what research is supported by public 
money’ and ‘making sure that the uses and purposes to which the research is put are responsible 
and allied with the laws and mores of our society’ — which includes ‘ethical oversight’ [P7]. 


Views about genomics research in general 

Four participants spontaneously raised the ‘ugly’ issue of eugenics and eugenic attitudes to genomic 
variations [P6] [P12]. A number of participants cited historical examples of genetic discrimination by 
regimes such as the Nazis as providing important learning for future genomics research. One 
participant cited the well documented historical precedent of the large technology company IBM 
being complicit in enabling regimes to carry out negative eugenics policies’”’, stating IBM were ‘the 
enablers for the Nazis ability to hunt down Jews and other "undesirables”’ [P6]. The same 
participant also raised concerns about contemporary and future ‘misuse’ of genomic data ‘for 
immigration’ [P6]. 
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Another participant raised concerns about deterministic applications of genomics for the early 
‘detection’ of mental health problems perceived as ‘undesirable’ to those with political power [P7]. 
One participant noted that political ‘demagogues’ can influence public opinion and as a result they 
‘generally trust qualified researchers and ethics committees’ and ‘trust “people” far less’ [P9]. 
Similarly, another participant stated ‘the more diverse the debate, the more dilute the effect of 
irrational preconception and ethical incompetence should become’ [P8]. 

Another participant raised the issue of pre-birth genetic testing, asking ‘isn't it less ethical not to 
present informed choice’, while acknowledging that the perception of impairment is subjective, and 
gave the example of the ‘Deaf and Hearing Impaired community’ not viewing deafness as ‘an 
impairment’ [P6]. The participant concluded with the question ‘can we speak for those who are 
different to us?’ [P6]. Another participant responded that ‘The ethics of pre birth testing will never 
be clear cut’ and cited the example of deaf couples to selecting ‘an embryo for IVF that would be 
deaf like them’ [P4]>*°. 

There was a recognition that different groups in society might have different interests and influence. 
The discussion also explored who should be involved and in which tasks. One participant articulated 
groups including ‘medical scientists’, ‘social scientists’, ‘psychologists’ and the general public as 
being groups which should influence research, but noted that not all ‘groups should have equal 
influence’ [P10]. One participant noted they didn’t feel ‘qualified’ to ‘comment on aspects of science 
itself’ but felt ‘strongly’ that they should be involved in ethical decisions and sharing personal 
experiences to help inform research [P5]. One participant asked ‘there will be many interested 
groups so which ones will be listened to?’ [P4]. 

One participant noted that being ‘highly educated’ was an enabler for involvement and that having a 
‘bit of time on their hands’ was also a enabler [P4]. 


Detailed Summary of Thematic Area 2: Participant views about proposed research with sibling group 


Participants recognised they were part of a ‘unusual cohort’ and suggested they would be useful to 
study [P4]. One participant stated they believed the sibling group ‘were part of a eugenics 
programme in some way’ [P4]. Participants shared multiple views about proposed research with the 
sibling group. One participant stated that ‘this field of study is so huge and our involvement would 
be a 'first' in many ways’ [P5]. Participants recognised the complexity of research on their 
‘community of shared interest’ owing to multiple variables and one participant stated research with 
the sibling group would be ‘enormously complicated’ owing to ‘confounding variables’ but would be 
‘worth the effort’ [P9]. Another participant added that they would ‘wholeheartedly support the 
involvement of the next generation’ in any future research with the sibling group and noted any 
study design should ensure new siblings and their offspring should be ensure they can ‘become part 
of the research’ [P5]. 

One participant noted ‘there is a world of difference between the idea of a study’ of the sibling 
group and other genomic research [P7]. Another participant concluded that ‘we are on the cusp of 
new forms of information , study and knowledge about ourselves’ and stated that in the context of 
genomic research ‘we are important in the grand scheme of things’ [P6]. In a follow-up survey one 
participant stated they ‘could not at all care whether my genomics are public or not. | do not see 
that my genome is a matter for privacy concerns’ but recognised that others may feel differently 
[P9]. 

One participant proposed that research with the sibling group would ‘not be trying to push a set 
agenda or profit financially’ [P5]. Another stated that ‘people who are not looking for personal gain’ 
should be involved, including those ‘who have a desire to improve quality of life and help us 
understand ourselves’ [P6]. One participant added that it is a ‘good idea to involve research subjects 
in formulating the research questions’ [P10]. Another asked ‘whatever format is decided upon who 
would decide on the points for discussion?, implying participants’ tasks should include deciding this 
[P4]. One participant suggested that members of the sibling group could ‘form and seek out 
participants for the Ethics Committee’ [P6]. 


448 


449 


One participant stated it was a ‘civic duty’ to participate in research and that research participants 
should be involved in formulating the research questions [P10]. Participants spontaneously 
suggested potential research topics for the group, some serious issues and some appeared to be 
more light-hearted comments made in jest. More serious topics included ‘mental health’ [P6] and 
pharmacogenomics [P4], with more light-hearted suggestions including ‘career choices’ and hobbies 
[P4]. One participant felt they should be involved in ‘seeking answers to old, or not yet thought of 
questions’ and ‘looking beyond the known into a murky unknown’ [P6]. 

Participants shared multiple views about what possible areas of research topics could be explored in 
studies they could participate in and how these could be conducted Participants spontaneously 
suggested potential research topics for the group, some serious and some more light-hearted 
comments made in jest. More serious topics included ‘mental health’ [P6] and pharmacogenomics 
[P4], with more light-hearted suggestions including ‘career choices’ and hobbies [P4]. One 
participant felt they should be involved in ‘seeking answers to old, or not yet thought of questions’ 
and ‘looking beyond the known into a murky unknown’ [P6]. 

One participant suggested genomics research ‘a subject which cries out for more public discussion 
by those who have been unexpectedly and deeply affected by genomics findings’ [P5]. Another 
participant suggested a study of ‘any intersection of phenotypes’ including ‘psychological and 
intellectual’ with other genomics data [P8]. The participant concluded with the question ‘I have 
wondered what unwelcome genes we might have inherited?’, citing their own experience of genetic 
testing for both Parkinson’s and Huntington’s disease [P4]. 

One participant reported they were ‘happy to contribute any information from my own genome that 
might be useful in scientific or forensic research’ [P9]. One participant suggested a study design 
where participants can ‘all participate or not as we wish’ [P5]. They also commented they were ‘OK 
to provide genetic material for further analysis’. [P5]. Another participant concluded the siblings 
should ‘contribute their DNA’ for research [P9] and warned people from thinking that ‘your DNA is 
“you” [P9]. 

A ‘group of “special interest” people involved in a group discussion’ was suggested by one 
participant as a ‘simple but effective method of encouraging debate’, if participants can ‘can dip in 
and out’ [P4]. They also stated group discussion would not work as ‘there are too many voices and 
some would be drowned out’ [P4]. This participant also stated ‘we will have differing preferences for 
how such a project should be organised’ and asked ‘would it be possible to agree on some ideas and 
then post them to the group’ in order to involve people in co-creating how they will be involved and 
participate [P4]. 

The participatory research method was described as ‘commendable’ [P9]. One participant stated 
‘moderated face to face discussions (through video if need be) remain the best method in my 
opinion for focussed outcomes and decisions with groups of people’ [P7}. Another participant stated 
that ‘most of our group would be able’ to use video-conferencing platforms [P5]. Face to face 
synchronous discussion was ruled out by another participant as there are ‘too many voices’ which 
are across multiple time-zones [P4]. Participants suggested the idea of using one to one interviews 
as way of involving people (including using telecommunications) [P5], however, one participant 
noted that one to one interviews can restrict discussion and ‘be quite straight jacketed with 
circumscribed questions’, compared to online discussions [P4]. 

In reference to future research with the sibling group one participant stated that ideally ‘we would 
be able to exert control over the use’ of data [P7]. 


Detailed Summary of Thematic Area 3: Participant views about the online discussion 


Participants reported their motivation for participating in the research was to help researchers and 
the sibling group. Three participants stated they were participating partly because they were 
interested in learning what their siblings think, and the opportunity to ‘think through’ and ‘discuss’ 
issues together [P4] [P6]. 

Participants were asked to comment on their experience of participating in this study. One 
participant mentioned they were ‘surprised about the number of participants’ in the study, stating 
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they thought more siblings would have participated [P6]. Others stated the experience was 
‘interesting’ and they ‘enjoyed thinking about the questions posed and reading the responses of 
others’ and the ‘perceptive comments’ of the Facilitator [P7] [P4]. One participant stated ‘it is 
commendable that there is a concern about participatory research’ in reference to the research 
methods used by the study team [P9]. However the participant also noted that in the early stages of 
participatory action research there was more discussion than action [P9]. 


Four participants reported specific things about the way this study was conducted that facilitated 
their involvement. One participant said the entire process was ‘assiduous’ and that the ‘intent of this 
project’ was ‘obviously thoughtful and interesting’ [P9]. One participant said the ‘system seemed to 
work well’ [P7]. Another added that being used to online platforms like Loomio, or having previous 
experience of similar platforms and ‘used to’ that way of communicating might facilitate 
involvement using that communication mode. One participant suggested an alternative discussion 
format where the participants discussed a thread for 2 days and then had a 3 day break before 
coming to another thread [P7]. 

Four participants reported specific things about the way this study was conducted that were barriers 
to their involvement. A discussion about boundaries revealed that some participants felt ‘avoiding 
topics which might trigger emotions which are stressful or unpleasant’ could be viewed as 
‘restrictive, even censorious’ [P7]. The pace of the discussions was mentioned as moving ‘too 
quickly’ with another adding ‘more time’ was needed and the study team should ‘reconsider the 
pace of the research’ [P7] [P4] [P5]. Two participants stated the ‘platform presented technical 
difficulties’ [P4] and that it was ‘complicated’ [P5]. One participant stated the ‘premise and the 
purpose of the study could be clearer’ and that the various discussion threads were ‘difficult to 
untangle sometimes’ [P7]. They also mentioned it was ‘hard to be able to guarantee to do this every 
day for a period’ and that not doing so meant they ‘got lost’ [P7]. Another participant added that ‘it’s 
a difficult subject to discuss in a vacuum, without real life examples’ [P4]. 


Detailed summary of all other thematic areas 


Finding out they are part of sibling group has been a positive experience 

The study team also noted that before the main discussion began, participants shared many 
personal experiences and reflections on finding out they were part of the extended family. For 
example four participants spontaneously reported that finding out they had siblings and being part 
of the siblings’ online community was a positive experience, with one considering themselves ‘lucky’ 
[P4]. Another participant added ‘I am so happy to be part of this [group]’ and that the experience 
‘has changed and enriched my life’ [P6]. They added ‘we are beyond fortunate, we have our group’ 
[P6]. Participants’ reactions also suggested the siblings group offered a form of support, and are in 
contrast to contemporary views of the 1950s where people feared that offspring finding out about 
their origins might cause ‘psychological injury’4°. 

People have responsibility to be involved in research 

One participant stated it was a ‘civic duty’ to participate in research and that research participants 
should be involved in formulating the research questions [P10]. 

Motivation for participation to help researchers and sibling group 

Participants reported their motivation for participating in the research was to help researchers and 
the sibling group. 

Uncertainty about what they can offer but happy to help 

Two participants stated that while they were happy to help with research, they were uncertain 
about what they could offer or did not believe they were ‘knowledgeable enough’ [P6]. 

Desire to improve situation for people affected by assisted conception 

Three participants stated they hoped future research would improve the situation for people 
affected by donor conception, as ‘the views of donor conceived people were not considered for 
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many years’ [P4]. One participant stated ‘there should be better and accessible structures in place to 
support people through the process of self-discovery’ [P5]. 

Interested in learning what other siblings think and discuss issues 

Three participants stated they were participating partly because they were interested in learning 
what their siblings think, and the opportunity to ‘think through’ and ‘discuss’ issues together [P4] 
[P6]. 

Concerns about power imbalance in research 

Seven participants shared multiple views about power imbalances in research, including concerns 
about bias, conflicting interests, data breaches and ‘hidden motives’ [P2], with genomic research 
being ‘used to perverted ends’ for both political and financial purposes [P2]. 

Control of knowledge and data 

Three participants challenged established forms of knowledge and data control, stating that if 
research is ‘publicly funded, it must be publicly available’ and challenging concepts around 
intellectual property and patent laws, and that ‘patenting new life forms is unethical’ [P9]. Another 
participant raised concerns about ‘sponsored facts’ and asserted that the opinions about genomics 
research ‘must be respected’ [P5]. 

Who decides who decides what is ethical? 

While some participants felt ‘trust’ towards existing scrutiny for research oversight [P5], ethics 
committees were identified as having ‘failings’, and one participant asked the question ‘who will 
decide who will be on the ethics committee?’ [P4]. One participant suggested that members of the 
sibling group could ‘form and seek out participants for the Ethics Committee’ [P6]. Another 
participant added ‘Il am not sure of the ethics process but it does seem a shame that more of us 
cannot participate’ [P5]. Another participant stated ‘I trust the scientists and the ethical committees’ 
[P9], while another noted that ‘the ethics of DNA research generally will continue to be of huge 
importance and will continue as a political issue, triggering new laws and regulations’ and raised a 
concern that ‘the law will not be able to keep up with the research - and we as members of the 
general public won't either’ [P7]. One participant noted that while people might be experts about a 
process or data, but that does not make them ‘moral guardians’ [P5]. They concluded ‘no single 
body (and that includes the church and the government) has a right to dictate moral guidelines’ [P5]. 
Genomics research used for political purposes 

While some participants indicated that all health policy decisions have political power associated 
with them, five participants raised a number of specific concerns about genomic research being mis- 
used for political purposes and ‘wicked ends’ [P5]. One participant stated research should not be 
used ‘for political purposes’ but indicated they did not believe this was easily prevented [P5], with 
another participant sharing the ‘realism about the likely corrupt use of genetic information for 
political and financial purposes’[P2]. A number of participants cited historical examples of genetic 
discrimination by regimes such as the Nazis as providing important learning for future genomics 
research. One participant cited the well documented historical precedent of the large technology 
company IBM being complicit in enabling regimes to carry out negative eugenics policies”, stating 
IBM were ‘the enablers for the Nazis ability to hunt down Jews and other "undesirables”’ [P6]. The 
same participant also raised concerns about contemporary and future ‘misuse’ of genomic data ‘for 
immigration’ [P6]. Another participant raised concerns about deterministic applications of genomics 
for the early ‘detection’ of mental health problems perceived as ‘undesirable’ to those with political 
power [P7]. 

Questioning giving power to experts ‘reinforces dependency on experts’ 

One participant noted that ‘blind fear and reinforces dependency on experts, at the expense of 
genuinely holistic solutions’ and that the people should challenge the ‘unquestioning faith in a kind 
of scientific determinism’ which genomics can encourage [P12]. Another participant felt experts 
‘need to drive research’ but they ‘cannot do it in vacuum’ as the public need them and they need the 
public [P5]. 
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Research for profit and ‘Bullying’ by ‘big pharma’ 

Six participants expressed concern about research for profit and those with financial interests 
influencing research. One participant stated they would be happy to participate in research but that 
they ‘would however be concerned if this data was ever shared for commercial purposes [P5]. 
Another participant noted they felt that certain pharmaceutical companies were responsible for 
‘bullying’, contributing to ‘disinformation; ignorance and inflexibility of medical and scientific 
professions’ [P12]. One participant proposed that research with the sibling group would ‘not be 
trying to push a set agenda or profit financially’ [P5]. Another stated that ‘people who are not 
looking for personal gain’ should be involved, including those ‘who have a desire to improve quality 
of life and help us understand ourselves’ [P6]. Another participant stated that ‘the people paying for 
the research will influence it’, noting that if it is ‘public money the gatekeepers will have the greatest 
influence’ [P7]. One participant said they find it ‘distasteful that a private company can benefit from 
public research and then withhold the data or charge for it’ [P9]. 

What contro! do participants have? 

One participant raised the question of ‘what type of control the subjects of the study were allowed’ 
[P7]. They posed the question ‘Can we be both subjects and supervisors’ and suggested that this 
would be easier at the start during the design phase but more challenging later on in the research 
cycle [P7]. 

Why do we do research? 

One participant stated that research is ‘for humankind’ [P9], with another going further and saying 
‘research is for every and any aspect of life’, including other forms of life [P5]. 

Who should be involved in research 

Participants stated that anyone should be involved in research, with a experts, people affected by 
the research directly and the public all sharing perspectives in the context of research carried out 
with ethical oversight. 

Anyone should be involved in research 

Six participants stated that anyone should be involved in research, with one participant stating 
‘everyone should have a voice not just scientists and researchers’ [P5]. Another participant stated ‘it 
needs to be a wide ranging discussion so that the benefits and possible problems can be fully 
explored’ [P4]. One participant said it can depend ‘what kind of research it is’ and what the purpose 
is [P7], with another adding ‘we all need to have a voice’ as ‘we may not be “experts” in genomics 
but our opinions must be respected and have validity’ [P5]. 

Those affected by research should be involved 

Six participants stated those affected by research should be involved, however others challenged 
this saying this could provide ‘a rather one eyed perspective’ [P4]. One participant stated ‘some 
research will benefit certain people - those people should probably influence it if they are an 
identifiable group’. Another participant noted that ‘we're all biased; whoever is affected by a 
condition is likely to want it prioritised’ [P12]. 

People with specific experience and skills should be involved 

One participant stated that people (including the public and research participants) will have a 
‘variety of professional and technical and creative skills’ which will be useful, with the most useful 
one being ‘knowing ourselves’ [P7]. 

The public and research participants should be involved in research 

One participant stated ‘I am a strong supporter of patient involvement in medical care’ and that 
‘involving members of the public’ in genomic research was important in order to ‘have their views, 
reactions, interpretations, questions, concerns sought, interacted with, and considered’ [P11]. 
Experts should be involved (over seen by ethics boards) 

Two participants stated that experts (including ‘scientists’ [P9]) who ‘know what they are doing’ 
should be involved [P2], with ‘research reviewed by ethics boards’ [P9]. One participant stated they 
‘loathe this current political atmosphere that is anti-intellectual, anti-expert, anti-science’ and that is 
why they believe that genomics research should be ‘left to the qualified’ [P9]. This view was 
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challenged by another participant who stated that it was often people who think they are qualified 
who in fact sometimes know ‘nothing at all’, for example regarding personal experiences ‘in practical 
terms’ [P5]. 

Which groups should have ‘equal influence’? 

There was a recognition that different groups in society might have different interests and influence. 
The discussion also explored who should be involved and in which tasks. One participant articulated 
groups including ‘medical scientists’, ‘social scientists’, ‘psychologists’ and the general public as 
being groups which should influence research, but noted that not all ‘groups should have equal 
influence’ [P10]. One participant noted they didn’t feel ‘qualified’ to ‘comment on aspects of science 
itself’ but felt ‘strongly’ that they should be involved in ethical decisions and sharing personal 
experiences to help inform research [P5]. One participant asked ‘there will be many interested 
groups so which ones will be listened to? adding that ‘It is easy to say only those people who are 
directly affected should have an influence on research but theirs could be a rather one eyed 
perspective’ [P4].’ [P4] One participant stated that ‘people who are not looking for personal gain, 
but who have a desire to improve quality of life and help us understand ourselves’ should influence 
research [P6]. 

Views on genomics research 

Participants shared views on genomics which demonstrated a high-level of medical and genomic 
literacy. One participant who is a qualified medical professional stated developments in genomics 
have ‘been the most significant of my lifetime’ [P8]. 

Another participant questioned the use of genomic medicine as an intervention before other 
solutions such as ‘simple lifestyle changes’ [P12]. They also questioned ‘a kind of scientific 
determinism’ which genomic research can encourage. 

Eugenics 

Four participants spontaneously raised the ‘ugly’ issue of eugenics and eugenic attitudes to genomic 
variations [P6] [P12]. One participant stated they believed the sibling group ‘were part of a eugenics 
programme in some way’ [P4]. Another participant raised the issue of pre-birth genetic testing, 
asking ‘isn't it less ethical not to present informed choice’, while acknowledging that the perception 
of impairment is subjective, and gave the example of the ‘Deaf and Hearing Impaired community’ 
not viewing deafness as ‘an impairment’ [P6]. The participant concluded with the question ‘can we 
speak for those who are different to us?’ [P6]. Another participant responded that ‘The ethics of pre 
birth testing will never be clear cut’ and cited the example of deaf couples to selecting ‘an embryo 
for IVF that would be deaf like them’ [P4]°8°. The participant concluded with the question ‘I have 
wondered what unwelcome genes we might have inherited?’, citing their own experience of genetic 
testing for both Parkinson’s and Huntington’s disease [P4]. 

Views on participation in genomics research 

One participant suggested a study design where participants can ‘all participate or not as we wish’ 
[P5]. They also commented they were ‘OK to provide genetic material for further analysis’. [P5]. 
Another participant concluded the siblings should ‘contribute their DNA’ for research [P9] and 
warned people from thinking that ‘your DNA is “you” [P9]. 

Research with siblings is unique and complex but important 

One participant stated research with the sibling group would be ‘enormously complicated’ owing to 
‘confounding variables’ but would be ‘worth the effort’ [P9]. Another participant added that they 
would ‘wholeheartedly support the involvement of the next generation’ in any future research with 
the sibling group and noted any study design should ensure new siblings and their offspring should 
be ensure they can ‘become part of the research’ [P5] . On participant noted ‘there is a world of 
difference between the idea of a study’ of the sibling group and other genomic research [P7]. 
Another participant concluded that ‘we are on the cusp of new forms of information , study and 
knowledge about ourselves’ and stated that in the context of genomic research. ‘we are important in 
the grand scheme of things’ [P6]. In a follow-up survey one participant stated the ‘could not at all 
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care whether my genomics are public or not. | do not see that my genome is a matter for privacy 
concerns’ but recognised that others may feel differently [P9]. 

Research topics — ‘into a murky unknown’ 

Participants spontaneously suggested potential research topics for the group, some serious and 
some more light-hearted comments made in jest. More serious topics included ‘mental health’ [P6] 
and pharmacogenomics [P4], with more light-hearted suggestions including ‘career choices’ and 
hobbies [P4]. One participant felt they should be involved in ‘seeking answers to old, or not yet 
thought of questions’ and ‘looking beyond the known into a murky unknown’ [P6]. 

One participant suggested genomics research ‘a subject which cries out for more public discussion 
by those who have been unexpectedly and deeply affected by genomics findings’ [P5]. Another 
participant suggested a study of ‘any intersection of phenotypes’ including ‘psychological and 
intellectual’ with other genomics data [P8]. One participant also indicated that involvement in 
research might help people make sense of their personal experience and added joining an online 
discussion ‘will let me think beyond my emotion’ [P6]. 

Choosing what to know about your genome 

Participants shared views which recognised there might be important variations in the knowledge 
people might chose to have about themselves and their genomes. One participant stated knowledge 
about whether someone was carrying a variation which pre-disposed them to Huntington’s could be 
a ‘poisoned chalice’ [P4]. 

Co-creating discussion boundaries 

In addition to inviting potential participants to help co-design discussions, participants were 
themselves invited to co-create their own boundaries for the group discussion by reviewing a 
suggested statement and suggesting amendments. The group was invited to self-creates code of 
conduct based on previous negative experiences of people unintentionally causing offence. One 
participant stated there was difficulty in self-censoring and knowing what others might find 
‘emotionally stressful or unpleasant’ [P7]. One participant stated ‘Common courtesy should be 
sufficient’ [P2]Another participant suggested ‘Perhaps we might agree that if one of us 
unintentionally makes a comment perceived as offensive by another that we agree either to 
apologise or explain our position’ [P5]. 

Experience of participating in research 

Participants were asked to comment on their experience of participating in this study. One 
participant mentioned they were ‘surprised about the number of participants’ in the study, stating 
they thought more siblings would have participated [P6]. Others stated the experience was 
‘interesting’ and they ‘enjoyed thinking about the questions posed and reading the responses of 
others’ and the ‘perceptive comments’ of the Facilitator [P7] [P4]. One participant stated ‘it is 
commendable that there is a concern about participatory research’ in reference to the research 
methods used by the study team [P9]. However the participant also noted that in the early stages of 
participatory action research there was more discussion than action [P9]. 

Enablers and barriers specific to this study 

Participants reported a number of enablers and barriers for involvement which were specific to this 
study. 

Enablers of participation and involvement 

Four participants reported specific things about the way this study was conducted that facilitated 
their involvement. One participant said the entire process was ‘assiduous’ and that the ‘intent of this 
project’ was ‘obviously thoughtful and interesting’ [P9]. One participant said the ‘system seemed to 
work well’ [P7]. Another added that being used to online platforms like Loomio, or having previous 
experience of similar platforms and ‘used to’ that way of communicating might facilitate 
involvement using that communication mode. One participant suggested an alternative discussion 
format where the participants discussed a thread for 2 days and then had a 3 day break before 
coming to another thread [P7]. The Facilitator (MC) noted that regular contact with the study team 
and timely support was essential and they ‘could not have done it without this’. 
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Barriers of participation and involvement 

Four participants reported specific things about the way this study was conducted that were barriers 
to their involvement. A discussion about boundaries revealed that some participants felt ‘avoiding 
topics which might trigger emotions which are stressful or unpleasant’ could be viewed as 
‘restrictive, even censorious’ [P7]. The pace of the discussions was mentioned as moving ‘too 
quickly’ with another adding ‘more time’ was needed and the study team should ‘reconsider the 
pace of the research’ [P7] [P4] [P5]. Updates from the discussion were sent to participants according 
to their preferences, and one stated they ‘lost track of emails’ and were sometimes unsure if they 
were ‘responding to the right part’ [P6]. Two participants stated the ‘platform presented technical 
difficulties’ [P4] and that it was ‘complicated’ [P5]. One participant stated the ‘premise and the 
purpose of the study could be clearer’ and that the various discussion threads were ‘difficult to 
untangle sometimes’ [P7]. They also mentioned it was ‘hard to be able to guarantee to do this every 
day for a period’ and that not doing so meant they ‘got lost’ [P7]. Another participant added that ‘it’s 
a difficult subject to discuss in a vacuum, without real life examples’ [P4]. One participant expressed 
‘trepidation’ at sharing views about research and compared the feeling to getting an answer wrong 
in an ‘exam’ [P6]. 

The Facilitator (MC) stated that they felt more time was required in the co-design process. In 
addition, the administrative processes surrounding the unplanned change of Chief Investigator and 
related administrative processes in relation to the ethics process (outside of the control of the study 
team) meant they felt support was ‘non-existent’ and was ‘wholly inadequate’ for the participatory 
research process being used. 


Emergent themes from qualitative investigator discussion data 


A number of themes were identified during a qualitative thematic analysis of the discussion. In order 
to maintain confidentiality, comments have not been attributed to individual facilitators and have 
been shared with the permission of those who participated in discussions. 


A personal or professional perspective? 


Facilitators reported finding it a challenge to separate ‘personal’ experiences from ‘professional 
perspectives’ when facilitating. The platform was new to all facilitators and a considerable amount 
of time was required in order to both train and support facilitators using the platform. While some 
issues were platform specific, any such platform will require training and ongoing support for those 
new to using it. This includes real-time video and voice calls. 
Facilitators required guidance on using personal experience, with the confidential investigator group 
serving as a place for advice and support. Future research of this kind should ensure that facilitators 
are supported appropriately, including being part of an active and confidential community of 
practice in order to give and receive practical, emotional and technical support. 
One members of the study team noted that facilitation is ‘not a dispassionate neutral and purely 
intellectual’ activity, and that some kind of personal motivation for facilitation can be helpful. This 
aligns with findings from the online discussion with participants, where the personal experience of 
being part of the sibling group was hard to separate from general views about genomics research. 
When analysing the data, one Facilitator noted ‘subjectivity isn't something shameful to be avoided - 
it's something to be acknowledged’. They later added: 
‘I think this discussion has also explored the boundaries of 'researcher'/'neutral 
Facilitator'/'person with skin in the game' - stepping back, they are all social constructions - 
some defined by law/ethics - others by un-codified conventions - and labelling them all is 
quite a liberating process.’ 


Critical mass 


Facilitators noted a ‘critical mass’ effect in online discussions, with the pace of comments seemingly 
affected by number of posts. The rate of posts to a discussion would increase exponentially, with 
more comments seeming to generate more comments in a ‘snow-ball’ effect. Conversely, 
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discussions with fewer comments and infrequent posts attracted fewer replies, with ‘energy levels’ 
appearing to ‘drop’. Facilitators theorised people were more confident to post in forums if there 
were more people posting, so they were less ‘exposed’. This aligns with findings from other studies 
which have explored participants’ hesitancy in posting in online discussion forums*+. 


Discussion pacing and participant engagement 


When co-designing discussions, the study team attempted to balance discussion pacing between 
being too slow and losing engagement and being too fast and overwhelming participants. After the 
discussions one Facilitator reflected they feared that ‘introducing a new thread might interrupt the 
flow’ and that it was difficult to judge this using this with online discussions. 
Facilitators also reported struggling to balance keeping people engaged and trying to get hesitant 
participants (people logging in, reading, but not commenting) to comment. Similarly, some active 
participants in discussions would not post for a number of days. Follow-up survey data from 
participants suggested that sometimes they could not post owing to other commitments, but the 
nature of online discussions meant they could catch up and join in when they had the time. 
One Facilitator noted that they underestimated how long it would take for people to get ‘properly 
signed up’, including giving technical support to participants. This created a tension when they 
‘wanted the latecomers to have time to ‘catch up' without being overwhelmed’. Another Facilitator 
also commented that they felt it took a week for trust to be established in the group, for example, 
for people to see that the co-created rules of the discussion were observed by everyone. 
Another Facilitator noted that: 
‘an advantage of this kind of discussion is that you can have those simultaneous discussions 
- yes people have a finite read/write/processing time in a day - but it allows those with 
interests in specific areas to pursue that without feeling left out, as they might do with a 
more traditional linear time discussion (like face to face or synchronous webinars)’ 
They also noted it allowed people to ‘refer back’ to previous discussions, making them more rich. 
Participants also reported enjoying having time to reflect on comments and reconsider views in light 
of others’ comments. 


Prescribed discussion and open-ended discussion 


One investigator noted the difficulty in achieving ‘the balance of being prescriptive (for consistency) 
and giving freedom to Facilitators’ to initiate discussions and follow emergent themes. Participants 
reported that they enjoyed being able to raise issues and that this was preferable over the more 
traditional scripted interviews, as it allowed areas to be discussed that researchers might not have 
considered. Facilitators agreed that using the ‘forking’ function on Loomio to ‘fork’ discussions 
moving in different directions into threads should be avoided as might confuse both participants and 
facilitators. 


Ethical limitations for participatory research 


The entire study team agreed that limitations in the ethics process affected the extent of how the 
sibling group could be involved in the study. Internationally, confusion still surrounds what ethical 
approval is required before involving potential participants in co-designing research as ‘specialist 
advisors’, with some guidelines now emmerging?2’. On the advice of the La Trobe University 
Human Research Ethics Committee, we did not approach potential participants about co-designing 
the study until after ethics approval had been granted, with feedback from participants being 
incorporated by a number of subsequent modifications to the original ethics application. As a result 
of the complex process of modifications, the timeline for feedback was shorter than the study team 
had anticipated, although the process did provide useful feedback!*3. Ambiguous policies for the 
ethical involvement of people in co-designing research can hamper the degree of control potential 
participants have in research and further clarity from ethics committees will enhance power sharing 
at this crucial stage of research. 
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Impacts from the process: Detailed summary 
Eight specific impacts were reported from this process. These are summarised in detail below. 
Impact 1: Improved understanding of genomics 


While participants showed a good-baseline level understanding of genomics, three reported their 
understanding about genomics and research increased as a result of participating in the study. 
However some participants demonstrated a lack of self-confidence in their understanding, in spite of 
demonstrating a good understanding of the principles of genomic research, citing relevant peer- 
reviewed literature in discussions and discussing the nuances of ethical oversight. 


Impact 2: Learning resources useful 


Participants reported finding the information resources and videos useful. One participant 
commented they had learned from the visual summary of the review of public involvement in 
genomic research stating ‘I didn't realise there were so many research projects involving global 
genomics projects but it is so good to read that public involvement is becoming more and more 
important’ [P5] However one participant stated ‘I must say that personally | hate being required to 
watch videos but think | am in a minority’ [P4]. 


Impact 3: Changed views and perspectives as a result of participating 


Four out of the six participants who completed the follow up survey noted their views and 
perspectives changed as a result of participating. One participant stated ‘I now realise how fast the 
field of genomics is changing and there are all kinds of implications especially in the field of precision 
medicine’ [P5]. One participant also indicated that involvement in research might help people make 
sense of their personal experience and added joining an online discussion ‘let me think beyond my 
emotion’ [P6]. Another stated ‘involving people in genomic research is crucial’ as it has ‘unknown 
consequences and needs as wider discussion as possible’[P4]. A number of participants had views 
about ‘leaving research to the qualified’ challenged by other participants [P9], with follow-up survey 
data suggesting that those challenged changed their views about who should be involved, towards 
widening. The changed views of the participants involved can be viewed as an impact of 
‘transformative learning’ 17. 


Impact 4:Participants asked to stay involved in the research 


All participants who completed the follow-up survey requested to stay involved in the research 
process, including in analysing data and being co-authors on the paper. 


Impact 5: Participants enjoyed the online discussions 


Participants stated the experience of participating was ‘interesting’ and they ‘enjoyed thinking 
about the questions posed and reading the responses of others’ and the ‘perceptive comments’ of 
the Facilitator [P7] [P4]. Another participant added that it ‘work well’ as a way of involving people 
[P7]. 


Impact 6: Improved understanding of how to get involved in research 


Participants reported improved understanding of how to get involved in research. However some 
participants demonstrated a lack of self-confidence in their usefulness in being involved in genomics 
research. Learning and development interventions to help people understand the valuable tasks 
they can complete in the research process might improve confidence. 


Impact 7: Co-design changed study design 


Feedback from participants resulted in changes to the study design including improving language 
used in recruitment, improving the online discussions and learning resources. 
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Impact 8: Method for future research co-design established 


By co-creating methods of involving participants in proposed future genomics research, this process 
has demonstrated a practical and well-evaluated method of involving potential participants in co- 
designing research. Participants stated that the methods used in this process could be helpful when 
co-designing future stages of proposed genomic research with the sibling group. 


Additional File 2: Standardised Data on Initiatives (STARDIT) report — Alpha 
Version: Co-designing genomics research with a large group of donor- 


conceived siblings 


About this report 


This report uses the Standardised Data on Initiatives Alpha version (STARDIT)**. The ‘living’ STARDIT 


Beta version? report which relates to this project can be found in the references 
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STARDIT Report Alpha Version 


Identifying information 


Initiative name 


Co-designing genomics research with donor -conceived siblings 


Geographic location or scope 


Australia 


Date range (planned start and 
end dates of initiative) 


2017-2020 


Purpose of the initiative 


Participatory action research to involve members of a sibling group in 
online discussions about how they would like to be involved in future 
research. 


Organisations or other 
initiatives involved (list all if 
multi-centre) 


4. School of Psychology and Public Health, La Trobe University 


Funding sources 


School of Psychology and Public Health, La Trobe University 


Clinical trial registration 
details (if applicable) 


N/A 


Ethics approval (if applicable) 


La Trobe University 


Other relevant information 
(free text) 


This report describes involving members of a sibling group in online 
discussions about how they would like to be involved in future research. 


At which stage of the research 
project has this report been 
written? (Select from: 

4. Before the 
intervention or 
initiative— this report 
is prospective or 
describes planned 
activity 

5. Ongoing -the 
intervention or 
initiative is still taking 
place 

6. After the research 
project or initiative 
has occurred 


After the participatory action research occurred. 
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Methods of the initiative 
(what is planned to be done, 
or is being reported as done) 


Name 


The research process was co-designed using a participatory action 
research method to involve members of a sibling group in the co-design 
of online discussions to explore future genomic research with members 
of the group. 


: Jack Nunn 

: Marilyn Crawshaw 
: Paul Lacaze 

: Shirley Brailey 


: Adrianne Smith 
: Barry Stevens 


Public domain profiles, 
institutional pages 


: https://scholars.latrobe.edu.au/display/j2nunn 
https://www.york.ac.uk/spsw/staff/emeritus-and-honorar 
crawshaw/ 
3: 
https: 
: N/A 
: N/A 
: N/A 
ttps: 


1 
2 
3 
4 
5: Barbara Nunn 
6 
7 
1 
2: 


marilyn- 


www.monash.edu/medicine/sphpm/about/staff/academic/lacaze 


www.wikidata.org/wiki/Q4864782 


Open Researcher and 
Contributor ID (orcid.org) 


= 


orcid.org/0000-0003-0316-3254 
orcid.org/0000-0002-2870-0506 
orcid.org/0000-0002-0902-6798 


ttps: 
ttps: 
: https: 


0) ES ES St Gy UY 
> JZ |z 


Tasks in report completion 


: Main report author 

: Checked report data 

: Checked report data 

: Checked report data and contributed additional data 
: Checked report data and contributed additional data 
: Checked report data and contributed additional data 
: Checked report data and contributed additional data 


NOM BWN RP 


Other information 


Other people were involved in writing this report and contributing data 
but did not want to be named. 


Key contact at initiative for 
confirming report content 

(include institutional email 
address) 


Who was involved or how 
would you label groupings of 
those involved 


Jack Nunn, PhD researcher, School of Psychology and Public Health, La 
Trobe University, jack.nunn@latrobe.edu 


Group 1: Academic research investigators (Jack Nunn, Marilyn Crawshaw 
and Paul Lacaze) 

Group 2: Members of the sibling group who gave feedback during the co- 
design stage (including but not limited to Becky Gardiner, David Gollancz 
and Michael Bywater) 

Group 3: Members of the sibling group who participated in the research 
and gave feedback as part of the co-design process, the manuscript 
checking stage or contributed data to the STARDIT report (including but 
not limited to Shirley Brailey, Barbara Nunn, Adrianne Smith and Barry 
Stevens) 
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How many people were in 
each grouping label? 


Group 1: 3 
Group 2:5 
Group 3: 18 


Specific tasks of this person or 
group (list as many as 
possible) — including any 
information about why certain 
people were included or 
excluded in certain tasks 


Group 1: Involved in co-designing every stage of the process, analysing 
data 

Group 2: Members of the sibling group were involved in refining wording 
of participant information, sharing views and advice about the process, 
proof-reading documents, providing feedback on surveys, analysing data, 
informing planning, and providing feedback on planned online 
discussions. 

Group 3: Participants were also involved in checking the content of 
Genetics Society UK podcast*®, with the recording shared with all 
participants before dissemination to ask them to check the content was 
accurate and acceptable. 

Group 2 and 3: Participants were sent the article and additional files to 
check the analysis and content and were invited to be authors of the 
STARDIT report. 


How were these people 
involved (what methods were 
used) 


Group 1: Face-to-face meetings, video calls, email communication, 
shared online documents, teleconferences. 

Group 2: video calls, email communication, shared online documents, 
teleconferences. 


Facilitators of involvement 
(what do you expect will help 
these people get involved — or 
what helped them get 
involved) 


Giving people time to read resources. Clear communication about the 
intention of involving people. 


One participant noted that being ‘highly educated’ was an enabler for 
involvement and that having a ‘bit of time on their hands’ was also an 
enabler [P4]. Being ‘respectful’ when involving ‘those affected by 
genomic research’ will facilitate research as the ‘more brains applied to 
research, the more likely answer to puzzles will be found’ [P11]. Similarly, 
another participant stated ‘the more diverse the debate, the more dilute 
the effect of irrational preconception and ethical incompetence should 
become’ [P8]. One participant stated that whatever model was chosen, it 
should be ‘as flexible as possible’ [P5]. 


Four participants reported specific things about the way this study was 
conducted that enabled their involvement. One participant said the 
entire process was ‘assiduous’ and that the ‘intent of this project’ was 
‘obviously thoughtful and interesting’[P9]. One participant said the 
‘system seemed to work well’ [P7]. Another added that being used to 
online platforms like Loomio, or having previous experience of similar 
platforms and ‘used to’ that way of communicating might facilitate 
involvement using that communication mode. One participant suggested 
an alternative discussion format where the participants discussed a 
thread for 2 days and then had a 3-day break before coming to another 
thread [P7]. 


The Facilitator (MC) noted that regular contact with the study team and 
timely support was essential and they ‘could not have done it without 
this’. 


Barriers of involvement (what 
do you expect will inhibit 


Face-to-face meetings were difficult to organise. The study team was 
located in both Australia and the UK, so face-to-face meetings were not 
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these people from getting 
involved — or what inhibited 
them from getting involved). 
Are there any known equity 
issues which may contribute? 


possible. Unclear communication about intentions and purpose of the 
involvement contributed to confusion (explaining how involvement is 
distinct from participation was challenging). Ensuring those involved had 
enough time to give feedback was also a challenge. 


Barriers to involvement in research identified by participants included 
public fear and ‘hysteria’ caused by a lack of understanding, which may 
‘hamper’ research, involvement, and general public support for research 
[P5]. Synchronous discussion was highlighted as another barrier if 
participants ‘are across time zones’ [P4]. One participant mentioned that 
they felt that their emotional response to some issues made it difficult to 
get involved in some ways [P6]. Being required to watch lengthy videos 
was identified as a barrier by one participant. One-to-one interviews 
were mentioned as being ‘quite straight-jacketed with circumscribed 
questions’ compared to more open online discussions [P4]. They also 
stated ‘I don't think that a group discussion would work as there are too 
many voices and some would be drowned out’ [P4] 


Four participants reported specific things about the way this study was 
conducted that were barriers to their involvement. A discussion about 
boundaries revealed that some participants felt ‘avoiding topics which 
might trigger emotions which are stressful or unpleasant’ could be 
viewed as ‘restrictive, even censorious’ [P7]. The pace of the discussions 
was mentioned as moving ‘too quickly’ with another adding ‘more time’ 
was needed and study team should ‘reconsider the pace of the research’ 
[P7] [P4] [P5]. Updates from the discussion were sent to participants 
according to their preferences, and one stated they ‘lost track of emails’ 
and were sometimes unsure if they were ‘responding to the right part’ 
[P6]. Two participants stated the ‘platform presented technical 
difficulties’ [P4] and that it was ‘complicated’ [P5]. One participant stated 
the ‘premise and the purpose of the study could be clearer’ and that the 
various discussion threads were ‘difficult to untangle sometimes’ [P7]. 
They also mentioned it was ‘hard to be able to guarantee to do this every 
day for a period’ and that not doing so meant they ‘got lost’ [P7]. Another 
participant added that ‘it’s a difficult subject to discuss in a vacuum, 
without real life examples’ [P4]. One participant expressed ‘trepidation’ 
at sharing views about research and compared the feeling to getting an 
answer wrong in an ‘exam’ [P6]. 


The Facilitator (MC) stated that they felt more time was required in the 
co-design process. 


What was the outcome or 
output of the involvement of 
these people? What changed 
as a result of involving 
people? 


Improved participant information resources, improved wording that was 
culturally appropriate (using terminology preferred by the sibling group 
to describe biological relations), improved online discussion, improved 
learning resources for participants, improved co-design process. 


At which stage of the initiative 
were these people involved? 
(select from list of pre-defined 
stages or allow ‘other’) 


Group 1: All stages 
Group 2: Co-design, evaluation and dissemination 
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What was the estimated 
financial cost for involving 
people. How much time did it 
take. Were there any costs 
that cannot be measured 
financially? 


$0 AUD — people volunteered their time. The total number of hours 
volunteered (excluding participation and the contributions of co- 
investigator MC) is estimated to be 25. 


What worked well, what could 
have been improved? Was 
anything learned from the 
process of involving these 
people? 


Describe any financial 
relationship or other interest 
this person has to this project 


Involving potential participants in co-defining language used to describe 
the sibling group helped ensure that language was acceptable and 
appropriate. 


The co-design process took longer than expected owing to ethical ‘grey 
areas’ with no clear instruction on whether ethics approval was required 
to involve people in co-design. As a result an ethics application was made 
and subsequent feedback from the co-design process was integrated 
using modifications to the ethics application. 


The entire study team agreed that limitations in the ethics process 
affected the extent of how the sibling group could be involved in the 
study. Internationally, confusion still surrounds what ethical approval is 
required before involving potential participants in co-designing research. 
On the advice of the La Trobe University Human Research Ethics 
Committee, the study team did not approach potential participants about 
co-designing the study until after ethics approval had been granted, with 
feedback from participants being incorporated by a number of 
subsequent modifications to the original ethics application. As a result of 
the complex process of modifications, the timeline for feedback was 
shorter than the study team had anticipated, although the process did 
provide useful feedback. 


Ambiguous policies for the ethical involvement of people in co-designing 
research can hamper the degree of control potential participants have in 
research and further clarity from ethics committees will enhance power 
sharing at this crucial stage of research. 


One investigator (Jack Nunn) is biologically related to participants from 
the sibling group, with one being his mother and all being half-aunts or 
uncles. 


Describe any conflicting or 
competing interests 


N/A 


Who is the data from this 
intervention shared with? 


It will be published open access in peer reviewed journals with identifying 
information removed in order to anonymise it as much as possible. 


How is it stored and hosted? 


It will be shared on a public domain repository. 


Who is analysing the data? 


Group 1: The study team described above 
Group 2: participants were invited to review the analysis and give 
feedback to ensure they felt it reflected their experience of the process 
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What methods will be used to 
analyse the data (including a 
link to any relevant code and 
information about validity) 


We used case study methodology to describe our experience involving 
participants in the co-design of the proposed study. We collected and 
analysed both qualitative and quantitative data during the involvement 
activities. 


We analysed data from online surveys and online discussions with 
participants. In addition, data from the study team communications was 
included, such as meeting notes, emails, reflexive diary entries and 
survey responses of study investigators. Coding and thematic analysis of 
qualitative data was carried out by two authors independently and 
checked by other authors. 


How is information about this 
data disseminated? 


10. It will be published in an open access journal 

11. It will be shared with participants of the research and also other 
members of the sibling group who have joined it since the study 
commenced 

12. Learning from this process has been presented at conferences, 
and will be shared on social media and through other channels. 
Preliminary learning was shared in a UK Genetics Society 
podcast**®, 


Who ‘owns’ the data or claims 
any kind of ‘intellectual 
property’ (include relevant 
licensing information) 


Confidential data collected as part of the study is stored according to 
laws and the data access plan approved by La Trobe University. 


The authors maintain ‘ownership’ of the data in the paper and is shared 
under the Creative Commons license used by the publishing journal. 


Who controls access to the 
data 


The study team, La Trobe University and participants will be involved in 
any future data access decisions. 


How is/will the data be 
‘Findable, Accessible, 
Interoperable, Reusable’ 
according to the FAIR criteria? 


What new knowledge has 
been generated? (if 
appropriate, include effect 
size, relevant statistics and 
level or evidence) 


Data will be shared in the public domain and licensed under a Creative 
Commons license. 


8. Involving participants in co-designing the research process 
resulted in a number of changes to the study design, including 
improving language used in recruitment and learning resources 

9. The process of involving people can be viewed as a learning 
experience for both the participants involved and study team 
members. The process changed participants’ views about who 
should be involved, which can be viewed as an impact of 
‘transformative learning’. 


What was learned 


Involving people in online discussions about involvement in research 
changes people’s views about who should be involved in research, 
including participants ‘widening’ their views about who should be 
involved in research to include more people. 


Knowledge translation 


3. Knowledge from this process will inform the design of a future 
genomic research 

4. Learning from this process can inform future involvement 
activities 
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Learning from this co-design process can inform future ways of 
involving people in genomic research including co-designing self- 
governed biobanks. 

Learning from this process was shared in a Genetics Society UK 
podcast, with the recording shared with all participants before 
dissemination to ensure the content was accurate and 
acceptable. 


Outcomes 


10. 


11. 


12. 


13. 


Co-design changed study design. Feedback from participants 
resulted in changes to the study design including improving 
language used in recruitment, improving the online discussions 
and learning resources. 

The process improved participants understanding about 
genomics and research. Participants had an improved 
understanding of genomics. While participants showed a good- 
baseline level understanding of genomics, three reported their 
understanding about genomics and research increased as a result 
of participating in the study. However, some participants 
demonstrated a lack of self-confidence in their understanding, in 
spite of demonstrating a good understanding of the principles of 
genomic research, citing relevant peer-reviewed literature in 
discussions and discussing the nuances of ethical oversight. 
Participants’ improved understanding about genomics and 
research helped them make informed decisions about 
invitations to join genomics research studies, which were shared 
with members of the sibling group after the process by 
researchers unconnected with this study. 

Learning from this process informed subsequent discussions in 
the sibling group about participation in research, including a 
proposed self-managed biobank. Method for future research co- 
design established. By co-creating methods of involving 
participants in proposed future genomics research, this process 
has demonstrated a practical and well-evaluated method of 
involving potential participants in co-designing research. 
Participants stated that the methods used in this process could 
be helpful when co-designing future stages of proposed genomic 
research with the sibling group. 

Participants reported finding the learning resources useful 
(including infographics and videos). One participant commented 
they had learned from the visual summary of the review of public 
involvement in genomic research stating ‘I didn't realise there 
were so many research projects involving global genomics 
projects but it is so good to read that public involvement is 
becoming more and more important’ [P5] 

Participants changed views and perspectives about genomics 
research as a result of participating. Four out of the six 
participants who completed the follow up survey noted their 
views and perspectives changed as a result of participating. One 
participant stated ‘I now realise how fast the field of genomics is 
changing and there are all kinds of implications especially in the 
field of precision medicine’ [P5]. One participant also indicated 
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14. 


15. 


that involvement in research might help people make sense of 
their personal experience and added joining an online discussion 
‘let me think beyond my emotion’ [P6]. Another stated ‘involving 
people in genomic research is crucial’ as it has ‘unknown 
consequences and needs as wider discussion as possible’[P4]. A 
number of participants had views about ‘leaving research to the 
qualified’ challenged by other participants [P9], with follow-up 
survey data suggesting that those challenged changed their views 
about who should be involved, towards widening. The changed 
views of the participants involved can be viewed as an impact of 
‘transformative learning’. 

Participants asked to stay involved in the research. All 
participants who completed the follow-up survey requested to 
stay involved in the research process, including in analysing data 
and being co-authors on the paper 

Participants enjoyed the online discussions. Participants stated 
the experience of participating was ‘interesting’ and they 
‘enjoyed thinking about the questions posed and reading the 
responses of others’ and the ‘perceptive comments’ of the 
Facilitator [P7] [P4]. Another participant added that it ‘worked 
well’ as a way of involving people [P7]. 


How has or how will this be 
measured? 


Future STARDIT reports 


Who is involved in measuring 
this? 


The study team and participants 
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Additional File 3 - GRIPP2 report: Co-designing genomics research with a large group of donor-conceived siblings 


This report has been completed using the ‘GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research’ available 


https://doi.org/10.1136/bmj.j3453. 
GRIPP2 short form 


Report the aim of PPI in the 
study 


Provide a clear description of 
2: Methods the methods used for PPI in the 
study 


Outcomes—Report the results 
of PPI in the study, including 
both positive and negative 
outcomes 


3: Study results 


Participatory action research to involve members of a sibling group in online discussions about 
how they would like to be involved in future research. 


The research process was co-designed using a participatory action research method to involve 
people from the sibling group in the co-design of online discussions to explore future genomic 
research with members of the group. Participants were also involved in checking the final 
version of the paper. 


Improved participant information resources, improved wording that was culturally appropriate 
(using terminology preferred by the sibling group to describe biological relations), improved 
online discussion, improved learning resources for participants, improved co-design process. 


Enablers of involvement: Four participants reported specific things about the way this study 
was conducted that enabled their involvement. One participant said the entire process was 
‘assiduous’ and that the ‘intent of this project’ was ‘obviously thoughtful and interesting’ [P9]. 
One participant said the ‘system seemed to work well’ [P7]. Another added that being used to 
online platforms like Loomio, or having previous experience of similar platforms and ‘used to’ 
that way of communicating might facilitate involvement using that communication mode. One 
participant suggested an alternative discussion format where the participants discussed a thread 
for 2 days and then had a 3-day break before coming to another thread [P7]. 


Barriers of involvement: Four participants reported specific things about the way this study was 
conducted that were barriers to their involvement. A discussion about boundaries revealed that 
some participants felt ‘avoiding topics which might trigger emotions which are stressful or 
unpleasant’ could be viewed as ‘restrictive, even censorious’ [P7]. The pace of the discussions 
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was mentioned as moving ‘too quickly’ with another adding ‘more time’ was needed and study 
team should ‘reconsider the pace of the research’ [P7] [P4] [P5]. Updates from the discussion 
were sent to participants according to their preferences, and one stated they ‘lost track of 
emails’ and were sometimes unsure if they were ‘responding to the right part’ [P6]. Two 
participants stated the ‘platform presented technical difficulties’ [P4] and that it was 
‘complicated’ [P5]. One participant stated the ‘premise and the purpose of the study could be 
clearer’ and that the various discussion threads were ‘difficult to untangle sometimes’ [P7]. They 
also mentioned it was ‘hard to be able to guarantee to do this every day for a period’ and that 
not doing so meant they ‘got lost’ [P7]. Another participant added that ‘it’s a difficult subject to 
discuss in a vacuum, without real life examples’ [P4]. One participant expressed ‘trepidation’ at 
sharing views about research and compared the feeling to getting an answer wrong in an ‘exam’ 
[P6]. 

The Facilitator (MC) stated that they felt more time was required in the co-design process. 


Involvement improved participant information resources, improved wording that was culturally 
appropriate (using terminology preferred by the group to describe themselves), improved online 
discussion, improved learning resources for participants, improved co-design process. 


Involving potential participants in co-defining language used to describe the group of people 
Outcomes—Comment on the affected helped ensure that language was acceptable and appropriate. 
4: Discussion and extent to which PPI influenced 
conclusions the study overall. Describe Involving participants in co-designing the research process resulted in a number of changes to 
positive and negative effects the study design, including improving language used in recruitment and learning resources. 


The process of involving people can be viewed as a learning experience for both the participants 
involved and study team members. The process changed participants’ views about who should 
be involved, which can be viewed as an impact of ‘transformative learning’. 


5: Reflections/critical |Comment critically on the study, | The co-design process took longer than expected owing to ethical ‘grey areas’ with no clear 
perspective reflecting on the things that instruction on whether ethics approval was required to involve people in co-design. As a result, 
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Section and topic 
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went well and those that did 
not, so others can learn from 
this experience 


an ethics application was made and subsequent feedback from the co-design process was 
integrated using modifications to the ethics application. 


Involving potential participants in co-defining language used to describe the group of people 
affected helped ensure that language was acceptable and appropriate. 


Involving participants in co-designing the research process resulted in a number of changes to 
the study design, including improving language used in recruitment and learning resources. 


The process of involving people can be viewed as a learning experience for both the participants 
involved and study team members. The process changed participants’ views about who should 
be involved, which can be viewed as an impact of ‘transformative learning’. 


Involving people in online discussions about involvement in research changes people’s views 
about who should be involved in research, including participants ‘widening’ their views about 
who should be involved in research to include more people. 
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Chapter 8 — Additional files: Involving Australian Indigenous 


peoples in precision medicine 


Prospective STARDIT Report: A Pathway to precision medicine for Aboriginal 


Australians 


This STARDIT report contains additional data relevant to the protocol ‘A pathway to precision 
medicine for Aboriginal Australians’. On online version of this report can be found here: 


Identifying information 


Initiative name 


Using Co-design to Enable Precision Medicine for 
Aboriginal Australians 


Geographic location or scope 


New South Wales, Australia 


Date range (planned start and end dates of 
initiative) 


Start: February 2021 
End: December 2022 


Purpose of the initiative 


The development of a new protocol using co-design 
methods to enhance the potential use of Precision 
Medicine for Aboriginal Australians. 


Organisations or other initiatives involved (list 
all if multi-centre) 


1. Poche Centre for Indigenous Health 
2. The National Centre for Indigenous Genomics 
(NCIG) 


Funding sources 


Poche Centre for Indigenous Health 


Clinical trial registration details (if applicable) 


Ethics approval (if applicable) 


Other relevant information (free text) 


This project is currently in the planning phase and 
seeking ethics approval before beginning the co- 
design phase. 


At which stage of the research project has this 
report been written? 


Before the initiative- this report is prospective and 
describes planned activity 


Methods of the initiative (what is planned to be 
done, or is being reported as done) 


This is an iterative qualitative study consisting of five 
main phases. In Phase-l, we will ensure appropriate 
governance of the project, establish a team of 
investigators and a Project Advisory Committee 
which includes Aboriginal community 
representatives. Following an initial consultation with 
the Aboriginal community, we will begin Phase-Il with 
the advertisement of the co-design workshops and 
invite community members to participate. In Phase- 
Ill, the Chief Investigators will participate in co-design 
workshops and document ideas generated during the 
workshops. The notes shall be subsequently analyzed 
thematically in Phase-IV with Aboriginal community 
representatives. Summaries of notes regarding 
participant recruitment, the consent process, DNA 
sample collection and storage, data governance and 
sovereignty, and the reporting of results and 
incidental findings are disseminated to the 
community. Lastly, in Phase-V we evaluate the co- 
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design process and adapt our protocol for the use in 
partnership with other communities. 


Report authorship 


Name 


1: Jack Nunn 
2: Kylie Gwynne 


Public domain profiles, institutional pages 


https://scholars.latrobe.edu.au/display/j2nunn 


2: https://researchers.mq.edu.au/en/persons/kylie- 
gwynne 


Open Researcher and Contributor ID (orcid.org) 


https://orcid.org/0000-0003-0316-3254 
https://orcid.org/O000-0002-6897-4528 


Tasks in report completion 


1: Main author of report 
2: Checked data of report 


Date of report authorship 


25% August 2020 


Key contact at initiative for confirming report 
content (include institutional email address) 


Jack Nunn, PhD researcher, School of Psychology and 
Public Health, La Trobe University, 


jack.nunn@latrobe.edu 


Involvement 


Who will be involved or how would you label 
groupings of those involved? 


Group 1: Academic investigators 

Group 2: Aboriginal community representatives 
Group 3: Aboriginal community members involved in 
co-design and consultation activities 


How many people will be in each grouping 
label? 


Group 1: 10 
Group 2:5 
Group 3: 20 


Specific tasks of this person or group (list as 
many as possible) — including any information 
about why certain people were included or 
excluded in certain tasks 


Group 1 and Group 2: Project design, ethics 
applications, planning and delivering co-design 
processes analysing data from co-design activities 


Group 3: Face to face and online events, consultation 
processes, checking data analysis 


How will these people be involved (what 
methods will be used) 


Group 1 and Group 2: formal investigator team, 
Project Advisory Committee, formal meetings, email 
discussions, commenting and editing documents, 
teleconferences, face to face meetings (when 
required) 


Group 3: Face to face and online events, co-design 
workshops and other activities to involve people 


Enablers of involvement (what do you expect 
will help these people get involved — or what 
helped them get involved) 


Co-designing planned involvement activities with 
Aboriginal community representatives will ensure 
they are appropriate, culturally safe and effective. 
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Barriers of involvement (what do you expect 
will inhibit these people from getting involved — 
or what inhibited them from getting involved). 
Are there any known equity issues which may 
contribute? 


Geographic isolation and COVID restrictions will make 
face to face meetings difficult. 


Access to IT equipment and reliable and affordable 
internet is not universal in Australia, including the 
Aboriginal communities we plan to work with. 


What was the outcome or output of the 
involvement of these people? What changed as 
a result of involving people? 


Involving Aboriginal community representatives and 
community members will ensure planned activities 
(including research methods) are appropriate, 
culturally safe and effective. 


involving people. How much time will it take. 
Are there any costs that cannot be measured 
financially? 


Which stage of the initiative will these people All stages 
be involved? (select from list of pre-defined 

stages or allow ‘other’) 

What is the estimated financial cost for $10,000 


Estimated time for planning and delivering 
involvement activity is 200 hours. 


The project is not planning to reimburse people for 
their time when involved (for example lost earnings if 
taking time off of work). The project may pay people 
for out of pocket expenses, or paid care which might 
be necessary for some people to be involved (for 
example, child care costs). 


Mapping financial or other ‘interests’ 


Describe any financial relationship or other 
interest this person has to this project 


Group 1: Academic investigators will be volunteering 
their time and may be named as authors in peer- 
reviewed publications 


Group 2 and 3: Aboriginal community representatives 
and community members may be paid for their time 


Describe any conflicting or competing interests 


N/A 


Data 


Who is the data from this intervention shared 
with? 


Data sharing (including returning data and results to 
participants), governance (including how to plan the 
management and storage of sample and DNA data) 
and data sovereignty will be agreed with the 
participants and the Aboriginal communities as part 
of the co-design process. 


The processes of data transference used in the 
project will be consistent with the principles of 
Participatory Action Research where stakeholders 
collectively decide upon roles, responsibilities and 
data access 


How is it stored and hosted? 


Stakeholders will collectively decide this as part of 
the co-design process, including stakeholder’s view 
on appropriate biobanking methods 


Who is analysing the data? 


Stakeholders will collectively decide this as part of 
the co-design process 
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What methods will be used to analyse the data 
(including a link to any relevant code and 
information about validity) 


Stakeholders will collectively decide this as part of 
the co-design process, including stakeholder’s view 
on appropriate integration of data into biobanks and 
other data repositories 


How is information about this data 
disseminated? 


1: Public domain websites 
2: Community events (online and face to face) 


Who ‘owns’ the data or claims any kind of 
‘intellectual property’ (include relevant 
licensing information) 


Stakeholders will collectively decide this as part of 
the co-design process, with a guarantee that 
Aboriginal community members will be involved in 
any final decisions 


Who controls access to the data 


Stakeholders will collectively decide this as part of 
the co-design process 


How is/will the data be ‘Findable, Accessible, 
Interoperable, Reusable’ according to the FAIR 
criteria? 


Stakeholders will collectively decide this as part of 
the co-design process 


What new knowledge do you expect to be 
generated? (if appropriate, include effect size, 
relevant statistics and level or evidence) 


1: Inform best practice on co-design with Aboriginal 
communities 

2: Answer important questions about people’s 
preferences about data ethics, security, and quality 
associated with genomic research 

3: Map people’s preferences using the STARDIT- 
Preference Mapping tool 


What do you hope to learn 


1: Community preferences about the development of 
a new genomic research protocol 

2: Community preferences about using co-design 
methods 


How do you hope this knowledge translation 
will be translated? 


1: Learning about community preferences and best 
practice can inform future co-design of genomic 
research and other initiatives with Aboriginal 
communities 


Outcomes 


1: Using a co-design process saves time, promotes 
the usage of health services, and elicits superior 
health outcomes 

2: Benefits specific to this PM project include the 
building of strong and committed community 
partners and enhancing skills and knowledge in the 
Aboriginal community about genomics and health 


How has or how will this be measured? 


Group 1, 2 and 3 will be involved in developing 
outcome measures of importance, collecting baseline 
data and follow up data and analysing any changes. 


Who is involved in measuring this? 


Group 1 and 2 
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Detailed baseline and follow-up data on preferences for 
involvement 


In the ausEE study, of the 26 people who gave consent and completed the initial survey, 15 
participated in the online discussion and 12 completed the follow-up survey. From the 12 responses 
to the baseline and follow-up survey, participants gave 41 responses. In the ausEE study 59% of 
participants’ responses showed a change towards ‘widening’ their view of who should be involved in 
research to include more people (N=24/41), 34% stayed the same (N=14/41) and 7% narrowed 
(N=3/41). 


In the Shared Ancestry study, from the six participants who completed both the baseline and follow- 
up surveys, a total of 54 responses were given. Of these, 35% showed a change towards ‘widening’ 
involvement (N=19/54), 8% ‘narrowed’ (N=8/54) and half stayed the same (N=27/54). 

This table combines the results from the 18 participants across the two studies who completed both 
the baseline and follow-up surveys using identifiable data at both stages, providing a total of 95 
responses to the questions. 45% of participants’ responses changed to ‘wider’ (N=43/95), 43% 
stayed the same (N=41/95) and 12% narrowed (N=11/95). 
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Study 


ausEE 
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Who should influence which aspects of research? 
Finding questions to ask 


Deciding which questions to prioritize and fund 


Deciding how to try and answer the question (the research method) 


Attempting to answer the question (carrying out the research, including 
collecting information) 


Trying to understand if it is possible to the answer the question (analysing the 
information) 


Sharing the information that has been found, and any answers that may have 
emerged (dissemination and publication) 


Ensuring that any information or answers are able to be used to help people in 
practice, policy or future research (sometimes called research translation) 


Change to wider 
3 


3 


No change 
1 


1 
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Change to narrower 
1 


1 


Study 


Shared 
Ancestry 
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Who should influence which aspects of research? 


Deciding if the way of asking the question and all the other stages of the 
research were appropriate (evaluating the research method and any impacts) 


Designing how people are involved in the research 


ausEE sub-totals (total of 41 responses) 
Finding questions to ask 


Deciding which questions to prioritize and fund 


Deciding how to try and answer the question (the research method) 


Attempting to answer the question (carrying out the research, including 
collecting information) 


Trying to understand if it is possible to the answer the question (analysing the 
information) 


Sharing the information that has been found, and any answers that may have 
emerged (dissemination and publication) 


Ensuring that any information or answers are able to be used to help people in 
practice, policy or future research (sometimes called research translation) 


Deciding if the way of asking the question and all the other stages of the 
research were appropriate (evaluating the research method and any impacts) 


Designing how people are involved in the research 
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Change to wider No change Change to narrower 
2 1 0 

2 1 0 

24 

2 

2 3 1 

2 2 2 

2 3 1 

3 3 0 

2 3 1 


N 
Ww 
pany 
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11 


12% of responses 
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Quantitative data from STARDIT-PM cross-case analysis 
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This table combines the STARDIT-PM data from all case studies in a cross-case analysis. Please note, data from the facilitators of both online discussions was 
combined, as the data source was a shared online discussion between facilitators from both the ausEE study and the Shared Ancestry study. In the ASPREE 
study, as data on interview participants was anonymised, we have no data on how many of the 59 interview participants were among the 18 who attended 
the event, so some people may have been counted twice. 


Online 

ASPREE ASPREE ASPREE facilitators (ausEE 
Interview event study team | ASPREE ausEE Shared ancestry | and Shared 

Data set participants | participants | members totals participants participants Ancestry) 

Number in 

dataset 20 18 4 42 26 12 3 

Views on who 

should be 

involved 10 18 3 31 17 10 0 

Views on who 

should do which 

tasks 5 18 3 26 10 8 0 

Views on modes 

of 

communication 13 18 1 32 3 4 0 

Views on what 

methods should 

be used 3 18 4 25 6 5 0 

Views on 

facilitators of 

involvement 3 18 4 25 10 7 3 

Views on 

barriers of 

involvement 8 0 2 10 5 6 3 
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Online 
ASPREE ASPREE ASPREE facilitators (ausEE 
Interview event study team | ASPREE ausEE Shared ancestry | and Shared 
Data set participants | participants | members totals participants participants Ancestry) 


Views on what 
the outcome or 
output of the 
research or 
involvement in 
research could 
be 3 4 4 11 3 2 0 
Views on which 
stage of the 
research people 


should be 
involved 2 18 4 24 0 1 0 
Research data 0 18 1 19 0 3 0 


Views on how 
learning from 
this research 

could be used 0 0 0 0 0 8 0 
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Detailed quantitative analysis of STARDIT data categories 
Table 12.1 summarises the quantitative data from the three case studies where it was reported (ausEE, Shared Ancestry, ASPREE), noting the Indigenous 
Precision Medicine case study is a prospective report. 


Table 12.1: 


Quantitative data from three case studies 
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Study 


Reported methods of 
involving people [STARDIT 
Data Category 'Method of 
doing task?'] 


Reported modes [STARDIT 
Data Category 
‘Communication modes] 


Reported tasks [STARDIT 
Data Category 'Tasks'] 


Involvement outcomes, impacts, learning or 
outputs [STARDIT Data Category ‘Involvement 
outcomes, impacts, learning or outputs'] 


ASPREE 


Commenting on 
documents, meetings, 
interviews, group 
discussion 


Total number: 4 


Face to face meeting, 
telephone interviews, face 
to face group discussion 


Total number: 3 


Reviewing documents 
(including participant 
information), commenting 
on research design, sharing 
views and perspectives, 
analysing data 


Total number: 4 


Summary of impacts on study design from 
involving people: 

1: Recruitment and sample collection plan 
changed 

2: Participant communication improved 

3: Participants will be involved in governance 
4: Participants involved in controlling data 
access 

5: Participants will be included on study 
advisory groups (including ethical oversight) 
using multiple communication modes 

6: Feedback will be provided to participants 
about the research 

7: Learning and development opportunities will 
be created for potential participants 


Total impacts: 7 
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Study Reported methods of Reported modes [STARDIT | Reported tasks [STARDIT Involvement outcomes, impacts, learning or 
involving people [STARDIT | Data Category Data Category 'Tasks'] outputs [STARDIT Data Category ‘Involvement 
Data Category 'Method of | 'Communication modes] outcomes, impacts, learning or outputs'] 
doing task?'] 

ausEE Commenting on Online shared documents, | Reviewing documents Summary of the six impacts reported from 


documents, survey, Group 
discussion 


Total number: 3 


online survey, online text- 
based asynchronous 
discussion 


Total number: 3 


(including participant 
information), commenting 
on research design, sharing 
views and perspectives, 
analysing data 


Total number: 4 


involving people: 

1. Participants reported learning resources 
were useful 

2. Participants reported changed views as a 
result of participating 

3. Participants reported enjoying the online 
discussions 

4. Online discussions to be used in future 
research prioritisation 

5. Participants asked to stay involved in the 
research 

6. Participants reported improved 
understanding of how to get involved in 
research 

Total impacts: 6 
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Study Reported methods of Reported modes [STARDIT | Reported tasks [STARDIT Involvement outcomes, impacts, learning or 
involving people [STARDIT | Data Category Data Category 'Tasks'] outputs [STARDIT Data Category ‘Involvement 
Data Category 'Method of | 'Communication modes] outcomes, impacts, learning or outputs'] 
doing task?'] 

Shared Commenting on Online shared documents, | Reviewing documents Summary of the eight impacts reported from 

Ancestry | documents, survey, Group | online survey, online text- (including participant involving people: 


discussion 


Total number: 3 


based asynchronous 
discussion 


Total number: 3 


information), commenting 
on research design, sharing 
views and perspectives, 
analysing data, 
contributing to STARDIT 
report data 


Total number: 5 


1. Improved understanding of genomics 
informed sibling's participation in future 
research 

2: Participants reported learning resources 
were useful 

3: Participants reported changed views as a 
result of participating 

4: Participants asked to stay involved in the 
research 

5: Participants enjoyed the online discussions 
6: Improved understanding of how to get 
involved in research 

7: Co-design changed study design 

8: Method for future research co-design 
established 

Total impacts: 8 
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Table 12.2: Cross-case analysis data from ausEE and Shared Ancestry case study 


Preference data 


Baseline - All aspects 
mentioned 


ee) 


Follow-up - All aspects 


N 


Baseline - Finding questions 
to ask (identifying research 
topics) 


Only people who are directly affected by the research 


Everyone (any member of the public who is interested) 
Only people who are participating in the research 


Anyone who might be indirectly affected by the research 


Only people with a professional role in research 
Anyone who might be indirectly affected by the research 


Follow-up - Finding questions 
to ask (identifying research 
topics) 


Everyone (any member of the public who is interested) 


Only people who are directly affected by the research 


Anyone who might be indirectly affected by the research 
i 
7 


| w 


N N N 
w AJP Total 
response 


Baseline - Deciding which 
questions to prioritize and 
fund 


Preference data 


Ancestry 


mT k 


; 


Follow-up - Deciding which 
questions to prioritize and 
fund 


N 


Only people who are directly affected by the research 
Baseline - Deciding how to try Onl a EEA h 
and answer the question (the nly people who are participating in the researc 


research method) 


Only people with a professional role in research 


Follow-up - Deciding how to 
try and answer the question 
(the research method) 


Ne 
NI] w 


Baseline -Attempting to 
answer the question 


(9) 
wn 
c 
=z 2 
So 
= L 
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Preference data 


Ancestry 


N 


Baseline - analyzing the 
information 


Only people who are directly affected by the research 
Only people who are participating in the research 
Only people with a professional role in research 
Anyone who might be indirectly affected by the research 
Everyone (any member of the public who is interested) 
Only people who are directly affected by the research 
Follow-up - analyzing the Only people who are participating in the research 
information Only people with a professional role in research 
Baseline - dissemination and | 24 Anyone who might be indirectly affected by the research 
publication 

Everyone (any member of the public who is interested) 
Only people who are directly affected by the research 


ray 
N 


(3) 
N 
c 
=z 2 
So 
=L 
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Preference data 


Follow-up - dissemination 
and publication 


Baseline - research 
translation 


Follow-up - research 
translation 


Baseline - evaluating the 
research method and any 
impacts 


486 


Ancestry 


N 


Only people with a professional role in research 
Anyone who might be indirectly affected by the research 
Everyone (any member of the public who is interested) 


Only people who are directly affected by the research 
Only people who are participating in the research 


N 


v 
c 
ve 
e n 
L 
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Preference data 


Follow-up - evaluating the 
research method and any 
impacts 


Baseline - Designing how 
people are involved in the 
research 


10 


Ancestry 


Only people with a professional role in research 4 JO 
Anyone who might be indirectly affected by the research 1 JO 
Everyone (any member of the public who is interested) 3 |0 
Only people who are directly affected by the research 1 J1 
Follow-up - Designing how Onl ee Heigstineinah h 1 Io 
peopleare involved inthe nly people who are participating in the researc 
research 10 Only people with a professional role in research 3 {1 
Total responses 268/84 
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STARDIT-PM preference mapping data from public survey for comparison 


This data below is included from a report about public preferences in science, shared by the charity Science For All.“ Identical STARDIT-PM data categories 
were used, allowing this data to be compared with data from this thesis, and combined with other data in the future. The table below summarises the 
answers to the question ‘Which aspects of any future research should be influenced by the following’. 


Deciding which Deciding how Attempting to Trying to Sharing the Ensuring that Deciding if the Designing how 

questions to to try and answer the understand if | information any way of asking people are 

prioritize and answer the question it is possible that has been | information or | the question involved in the 

fund question (the (carrying out the | to answer the | found, and answers are and all the research 

research research, question any answers able to be other stages of 
method) including (analyzing the | that may have | used to help the research 
collecting information) emerged people in were 
information) (dissemination | practice, appropriate 
and policy or 
publication) future 
research 

Everyone (any 5 7 8 3 15 1 8 6 
member of the 
public who is 
interested) 
Anyone who 4 2 2 5 0 10 1 2 
might be 
indirectly affected 
by the research 
Only people who 1 2 1 1 1 1 2 3 
are directly 
affected by the 
research 
Only people with 6 5 5 6 1 4 8 4 
a professional 
role in research 
Only people who |2 3 2 3 2 3 0 4 
are participating 
in the research 
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Discussion 


Strengths and limitations of methods used — detailed analysis 
Evaluation of review methods 


Narrative review of systematic reviews: How are the public involved in health 
research and what are the impacts? 


As part of this thesis, | completed a narrative review of systematic reviews about involvement. The 
conclusion of many of the systematic reviews found, and thus the conclusion of the narrative review 
was, that there is not currently enough data to complete a meta-analysis of quantitative or 
qualitative data about involvement in genomics research. | published an article in the WikiJournal of 
Medicine which summarised and compared over 30 different types of systematic reviews?®, and 
concluded a systematic scoping review was the most appropriate method to search for relevant 
data. Findings from this review suggested that methods of involving people guided by the paradigm 
of participatory action research were most likely to have impacts. This finding had significant 
implications for this thesis, as this paradigm was used to guide the entire thesis as a result of this 
review. 


Public Involvement in Global Genomics Research: A Scoping Review 


While the scoping review method was not an exhaustive summary of all data, working with a 
database provided by the Global Alliance for Genomics and Health (GA4GH) provided a useful 
‘snapshot’ of current international genomics research projects?’. However, while the database 
compiled by GA4GH contained a majority of the predominant genomics research initiatives globally, 
the authors were aware of initiatives not represented, and a subsequent update after the 
completion of the review added 125 new initiatives”, creating an impetus for an updated review. 
Furthermore, the review extracted data at the level projects articulated in the database and did not 
analyse the structure of organisations which sat above multiple projects. For example, the Broad 
Institute had a number of projects that were included in the database and a number that were not. 


Another limitation of the scoping review search process was related to the wider issue that concepts 
such as ‘involvement’ cannot always be expressed fully in linguistic constructs even within the 
English language, let alone other languages. Such limits on English language reporting were explored 
in my presentation about participatory action research for the German-language organisation 
Ludwig Boltzmann Gesellschaft (LBG)*9"4". As a result of such imprecise language, a decision on 
whether an initiative was reporting involvement sometimes required individual judgement. While 
the authors attempted to overcome this by using objective ‘involvement indicators’, it is never 
entirely possible to remove such individual judgement in this kind of review, so such decisions were 
checked by multiple authors?*”. The use of similar indicators in STARDIT was an attempt to combine 
individual judgements in order that multiple perspectives can be incorporated*””. While a move 
towards more concrete and objective indicators of involvement will bring more consistency to 
definitions of involvement, truly ‘standardised’ descriptions will never be possible. By using Wikidata 
(which can have definitions updated by anyone at any time in any language), STARDIT can also be 
used to incorporate evolving language as it changes to reflect developments in different disciplines. 


Another significant limitation of the scoping review was finding reports of involvement that gave a 
definite answer to the question ‘did the involvement influence the research?’. For example, it is best 
practice that deliberative public dialogue and consultation should report the consequences of 
involving people 22P23)32736), That is, there is an assumption that if people are asked to share views 
and perspectives, any new knowledge generated from such a process will be used to influence the 
genomics research initiative. While there were many initiatives that reported using methods where 
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tasks included people sharing views and perspectives, only a few directly reported that these 
influenced the research (for example, the UK Biobank Ethics Consultation Workshop)***. The issue of 
deciding if a method of involving people was ‘consequential’ was not possible to resolve. If 
involvement is ‘consequential’, it means that the process of involving people contributing to 
influencing research process, as distinct from involvement which is ignored or not incorporate 
328 It was not always possible to determine whether involvement was consequential based on the 
available information, so an assumption was made that all methods reported resulted were 
‘consequential’. 


326- 


STARDIT provides a function for initiatives to report any impacts from involvement, as well as the 
ability to update a report throughout the project lifetime, thus allowing the consequences of 
involvement to be reported into the future. With some genomics research projects projected to 
span decades’, such reporting will provide valuable data about how involvement has impacted and 
influenced genomics research projects. The uptake of STARDIT to report the Australian Genomics 
project ‘Involve Australia’, including impacts from involving people?®, demonstrates the perceived 
viability of such a method. 


While this scoping review was the first of its kind in this field, a subsequent review exploring 
reporting the impact of involvement in biobanks used a similar method and recommended the use 
of STARDIT for reporting the impact of any involvement®*. The choice by this team to use a similar 
method validates the choice of a scoping review methods, and STARDIT as a system used for 
reporting such data. 


While future systematic reviews exploring involvement in genomics research and any impacts would 
greatly improve understanding in this area, at present, there is not enough data in order to complete 
such a review type. If STARDIT is used to report multiple genomics research projects, including 
involvement in those projects, it will provide standardised data from which future systematic 
reviews can be created, including living systematic reviews>®’. As the field of involvement in 
genomics research is growing fast, living systematic reviews provide a practical way for those 
planning involvement in genomics research to evaluate data and make evidence informed decisions 
about involvement. 


Guidance for planning, reporting and evaluating initiatives: A multidisciplinary 
scoping review 


As per the narrative review of systematic reviews of public involvement, this review explored 
guidance for planning, reporting and evaluating initiatives beyond the scope of genomics research in 
order to find relevant models and frameworks?”'. The co-design process of STARDIT meant that the 
system was developed into a way or reporting involvement across multiple disciplines including 
genomics research. This review attempted to show the current variation in guidance on planning, 
reporting and evaluating initiatives in order to inform both future systematic reviews and proposed 
standardised ways of reporting data on initiatives. 


The review showed the multiple different ways of reporting data within and across disciplines, and 
informed the co-design of STARDIT, ensuring it was built in a way to incorporate the multiple 
different reporting standards which exist. 


Evaluation of online discussion method 


While the research projects described in the case studies in this doctoral research were planned and 
completed before the COVID-19 pandemic, the methods of involving people online described in this 
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thesis now have an unexpected relevance to many disciplines, as research projects around the world 
seek to involve people online in novel ways, and evaluate such methods in a standard way. 


The importance of safe online spaces is being recognised as more important than ever, with the 
concept of safety extending to ‘data privacy’ and ‘cultural safety’. The ASPREE study showed that 
people preferred not to use social media companies to get involved in research. Using code from an 
open-source version of Loomio, the online forum was installed and hosted on La Trobe University 
virtual machines running on Australian Government servers, which provided reassurance to 
participants that the data would be stored ethically and in alignment with their values. This would 
not have been possible if using many third-party platforms. 


The online text-based discussion method itself provided a flexible way for people in different time 
zones to get involved and interact. Hosting it as an asynchronous (people do not need to be online at 
the same time) discussion over two weeks also provided flexibility for those with caring or other 
responsibilities, allowing them to participate at times which suited them, without feeling excluded. 
The decision to use an asynchronous text-based discussion for two case studies was vindicated by 
the positive experience reported by participants. Literature suggested it was a more inclusive 
method, as it supports people with different cognitive needs or preferences?”*. This was echoed by a 
comment from an ASPREE participant at a face to face event, who was concerned that more 
dominant, confident or cognitively able voices would be heard in group discussions. While this was 
mitigated by facilitating smaller group discussions, the online discussions successfully created the 
enabling conditions for everyone to be heard equally and thanked individually for their contribution. 


The facilitators reported that the support they received improved their ability to facilitated 
discussions, as did the shared online discussion space for the facilitators from both the Shared 
Ancestry study and the aausEE study both. The decision to create this was informed by the reflective 
practice articulated in the methods section?™. 


Owing to how data was reported and shared (including using STARDIT), such learnings from these 
online discussions can be used to inform others who are planning similar methods™®. Such learning 
can also inform how siblings, people with recent shared ancestry and other people at greater risk of 
exploitation could actually achieve workable methods to create power-sharing structures in line with 
the participatory action research paradigm, including to reporting any such methods in a standard 
way to inform future involvement. 


Learning from this project also influenced how the charity ‘Science For All’ was set up and 
established by myself and others in parallel with this doctoral research (see ‘Thesis Timeline’). 
Learning from these case studies informed how Science for All used Loomio to self-organise and run 
collaborative research projects with multiple stakeholders?*®, including the co-creation process for 
STARDIT*. 


Evaluation of the case study method and cross-case analysis 


The case studies selected for this doctoral research represent four unique and real-world 
communities of people, where genomics research affects their lives directly. By ensuring that 
communities of people affected were involved in shaping this research, it has helped ensure learning 
is anchored in reality, rather than theoretical models of involvement in genomics research. 


The case study methods used in this thesis, guided by the paradigms described, allowed the 
collection and analysis of both qualitative and quantitative data from multiple sources, which 
provided a richer dataset. In addition, the collection of data from study team members and 
participants provided a more holistic perspective, and meant data and impacts were collected that 


491 


492 


would not have been if data was just collected from participants. The collaborative analysis which 
was achieved by involving multiple stakeholders also ensured multiple people were involved in 
checking the analysis reflected their experience of the research process. The reflective analysis was 
especially important during my analysis of data from the Shared Ancestry, as | am directly related to 
participants. This was able to be balanced and compared with the experiences of other study team 
members. 


The sample sizes for the case studies in this thesis was variable, as was the percentage of people 
recruited from the known populations. While the ASPREE case study was relatively large?” involving 
59 participants and 20 interviews analysed, the total number of people was a relatively small sample 
from the 14,268 ASPREE participants (N=59/14268, 0.4%). The ausEE case study was medium sized 
with 29 participants, representing over 1% of people who were recruited from an online community 
of around 2000 people (N=29/2000, 1.45%). While the Shared Ancestry case study was a relatively 
small size compared to the other two (12 gave consent and 6 actively participated in the online 
discussion and surveys), it was recruiting from total known population of 18 at the time. Counting 
the two siblings [P8-SA, P10-SA] that were involved in the co-design process (but that did not 
participate in discussions), this gives a total of 15 out of 18 involved in some capacity (N=15/17, 
88%), which is the highest proportion of all the case studies. It is interesting to note that after this 
study ended, there are now 46 known siblings (some deceased), as of 12'* October 2021. Using 
STARDIT was an attempt to overcome this variation in both sample size and percentage of people 
recruited in the case studies, by combining standardised data. While datasets are still too small to 
draw any statistically significant conclusions, the mixed-methods approach meant that the 
interpretative analysis was able to combine themes and provide useful data, including comparing 
differences and generalisations. 


As previously noted, the selection of case studies was influenced by pragmatic considerations. It is 
important to note that the unselected case studies also occupied a ‘work-load’ grey area, with 
conversations with potential partner organisations identifying areas for support in the organisations 
| was approaching. However, the line between ‘research’ and providing a pro-bono service to the 
organisation required careful elucidation. Working within the structure of a PhD to provide such pro- 
bono services is problematic for ethical, practical and financial reasons. Establishing partnerships 
through the neutral and transparent structure of a charity makes navigating such grey-areas more 
practical. For example, pro-bono and paid work can be articulated in working agreements, as can 
any data shared for research and publication. This model of working was used for developing 
STARDIT, rather than presenting it as a case-study in itself. 


While the case studies were variable in size and each had both ‘representative features’ and ‘deviant 
features’ (see the section ‘Case selection’)*”“, the most significant learning for others planning 
genomics research was likely to be from the ‘generalisable’ features of the case studies. Accordingly 
the cross-case analysis provided a successful way to apply the post-positivist paradigm and explore 
common themes across the case studies, and combine the quantitative data to allow cross-case 
quantitative analysis. 


Evaluation of standardised data reporting method (STARDIT) 


This section is an evaluation how effective STARDIT was for enabling consistent data reporting and a 
cross-case analysis. It is not an evaluation of STARDIT itself. A discussion about the strengths and 
limitations of STARDIT can be found in the discussion section of the peer-reviewed section 
‘Standardised Data on Initiatives - STARDIT: Beta Version’®. 


The preference mapping tool (STARDIT-PM) allowed consistent mapping of different stakeholder 
preferences across all case studies where it was used. This included the preferences of both 
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potential participants and study team members. Applying a quantitative analysis to the identical 
questions asked at the start and the end of the ausEE case study and the Shared Ancestry study 
allowed an investigation of baseline preferences about involvement in genomics research and 
preferences after the online discussions. It showed how people’s preferences changed, including 
showing a ‘widening’ towards people preferring more kinds of stakeholders involved in genomic 
research. This data was published online in the public domain in a consistent way, allowing it to be 
combined in the cross-case analysis, and open to future datasets which might use the same question 
structure. 


The design and reporting cycle in STARDIT was also helpful to allow consistent mapping of the 
participatory action research process for the different case studies. The design cycle proposed in 
STARDIT was also adopted and used in the protocol for the Indigenous Precision Medicine project, 
including proposing using STARDIT reports at various stages of the research’. STARDIT also 
demonstrated a working Beta system for reporting involvement and any impacts and has been 
recommended for use in reporting involvement in biobanks*. It has also been adopted by the 
Australian Genomics project 'Involve Australia’, who created a prospective STARDIT report in 
20217"°. 


The impacts reported using STARDIT in this thesis were sometimes ‘transformative learning’. While it 
is often difficult to attach causality to learning interventions (such as the information resources or 
videos used in the case studies), STARDIT allows consistent data on this to be reported (including 
self-reported outcomes), facilitating any future analysis and allowing future statistical analysis to 
begin to draw any correlation between certain learning interventions and learning outcomes, which 
may suggest causality. 


STARDIT was also used by other projects outside the discipline of genomics research to report a 
participatory action research process involving citizen science and environmental DNA collection, 
demonstrating it can be used to allow comparison across case studies and beyond this doctoral 
research. 


Evaluation of outcomes and impact assessment methods 


While time for longer term impact assessment is not possible within time limits of a PhD, it was still 
possible to measure outcomes and impacts immediately after the participatory action research 
process. Some impacts were able to be measured over two years after the online discussions had 
finished, as participants from the Shared Ancestry study were involved in co-creating the Alpha 
version STARDIT report in October 2020, and able to edit the Beta version from August 2021 
onwards. These significant additional impacts about how participation in the participatory action 
research process helped participants make informed decisions about participating in genomics 
research would not have otherwise been recorded. 


A follow-up survey was conducted with the study team for the ASPREE study, but not the 


participants, owing to ethical restrictions. As a result, there may be multiple unreported impacts. 
Preference and follow-up data was also not collected for the ASPREE study for the same reasons. 
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About this document 


Usage and sharing 


This document is shared under the principles of the open access movement, and this document 
should always therefore be free of cost or other access barriers to everyone in the world. 


Everything in this thesis document is licensed under a CC BY 4.0 License by the author, Jack Nunn, 
(including content which is published in peer-reviewed journals with co-authors under an identical 
license), with the exception of Figure 8.2 ‘Precision Medicine project meeting participants 2019’, 
which is reproduced with permission from a photo shared in the public domain by the Poche Centre 
for Indigenous Health.: Separate permission should be sought to reproduce and share this image. 


The CC BY 4.0 License permits use, sharing, adaptation, distribution and reproduction in any medium 
or format, as long as you give appropriate credit to the original author(s) and the source, provide a 


link to the Creative Commons licence, and indicate if changes were made. 


Please note the license may not give you all of the permissions necessary for your intended use. For 
example, other rights such as publicity, privacy, or moral rights may limit how you use the material. 


Contact and identification 


For any questions, queries or feedback you can contact the author Jack Nunn by: 


e Emailing Jack.Nunn@Latrobe.edu.au or Jack.Nunn@ScienceForAll.World 
e Twitter: @JackNunn 


Jack Nunn’s ORCID is O000-0003-0316-3254, with contact information regularly updated via this 
account. 
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